AnsweredAssumed Answered

Virtualisation server doesn't retry connection

Question asked by sacco on Aug 3, 2007
Latest reply on Sep 29, 2008 by sacco
    Alfresco versions: any from 2.1.0RC1 through to latest nightly build of 2.1.0

    Using  alfresco-community-tomcat  to eliminate tomcat version problems.

    IBM JVM 1.5.0

    RHEL 4
The virtualisation server is clearly intended to retry the JMX remote connection until the
main Alfresco server comes up, but it doesn't!

It seems that the authenticationService fails to connect and then simply gives up. 
If I wait until the main server is up and then start it manually, then the Virtualisation
server can be started.

The advice given here:
http://forums.alfresco.com/viewtopic.php?t=5343&start=14
doesn't seem to hold: The virtualization server *does* gives up.

The point at which it gives up is just when it finds the remote port not yet open:
if I put the following into my virtual_alf.sh script to poll the port just before the line
which calls  startup.sh for the virtual-tomcat, then the Virtualisation server now sees
an open port and does not immediately give up.

while  ! 4</dev/tcp/${Alf_Server:=localhost}/${Alf_JMX_port:=50500} true \
   2>/dev/null  4<&- ;
  do  4<&-  sleep 1s ; done  4<&-  2>/dev/null

Here's a typical stack-trace:
INFO: The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/lib/jvm/java-1.5.0-ibm-1.5.0.5/jre/bin:/usr/lib/jvm/java-1.5.0-ibm-1.5.0.5/jre/bin:/usr/lib/jvm/java-1.5.0-ibm-1.5.0.5/jre/bin/classic:/usr/lib/jvm/java-1.5.0-ibm-1.5.0.5/jre/bin:/usr/lib
03-Aug-2007 01:34:52 org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-8180
03-Aug-2007 01:34:52 org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 3337 ms
03-Aug-2007 01:34:53 org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
03-Aug-2007 01:34:53 org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/5.5.23
03-Aug-2007 01:34:53 org.apache.catalina.core.StandardHost start
INFO: XML validation disabled
03-Aug-2007 01:34:56 org.apache.catalina.core.StandardContext start
SEVERE: Error filterStart
03-Aug-2007 01:34:56 org.apache.catalina.core.StandardContext start
SEVERE: Context [/manager] startup failed due to previous errors
03-Aug-2007 01:34:56 org.apache.catalina.core.StandardContext start
SEVERE: Error filterStart
03-Aug-2007 01:34:56 org.apache.catalina.core.StandardContext start
SEVERE: Context [/host-manager] startup failed due to previous errors
03-Aug-2007 01:34:56 org.apache.catalina.core.StandardContext start
SEVERE: Error filterStart
03-Aug-2007 01:34:56 org.apache.catalina.core.StandardContext start
SEVERE: Context [] startup failed due to previous errors
03-Aug-2007 01:34:56 org.springframework.core.CollectionFactory <clinit>
INFO: JDK 1.4+ collections available
03-Aug-2007 01:34:57 org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver-context.xml]
03-Aug-2007 01:34:57 org.springframework.context.support.AbstractRefreshableApplicationContext refreshBeanFactory
INFO: Bean factory for application context [org.springframework.context.support.FileSystemXmlApplicationContext;hashCode=1720215176]: org.springframework.beans.factory.support.DefaultListableBeanFactory defining beans [virtserver-properties,avmReauthenticatingAdvisor,reauthenticatingAdvice,clientTicketHolder,authenticationService,avmRemote,avmRemoteTransport,avmRemoteTransportRaw,attributeService,attributeServiceTransport,attributeServiceTransportRaw,jndiInfoBean,virtServerInfo,virtServerMBeanServer,virtServerRegistry,serverConnector,virtServerExporter,virtWebappRegistry,cacheControlInfo]; root of BeanFactory hierarchy
03-Aug-2007 01:34:57 org.springframework.context.support.AbstractApplicationContext refresh
INFO: 19 beans defined in application context [org.springframework.context.support.FileSystemXmlApplicationContext;hashCode=1720215176]
03-Aug-2007 01:34:58 org.springframework.core.io.support.PropertiesLoaderSupport loadProperties
INFO: Loading properties file from URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-shared.properties]
03-Aug-2007 01:34:58 org.springframework.core.io.support.PropertiesLoaderSupport loadProperties
INFO: Loading properties file from URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver.properties]
03-Aug-2007 01:34:58 org.springframework.context.support.AbstractApplicationContext initMessageSource
INFO: Unable to locate MessageSource with name 'messageSource': using default [org.springframework.context.support.DelegatingMessageSource@2fd42fd4]
03-Aug-2007 01:34:58 org.springframework.context.support.AbstractApplicationContext initApplicationEventMulticaster
INFO: Unable to locate ApplicationEventMulticaster with name 'applicationEventMulticaster': using default [org.springframework.context.event.SimpleApplicationEventMulticaster@7b287b28]
03-Aug-2007 01:34:58 org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in factory [org.springframework.beans.factory.support.DefaultListableBeanFactory defining beans [virtserver-properties,avmReauthenticatingAdvisor,reauthenticatingAdvice,clientTicketHolder,authenticationService,avmRemote,avmRemoteTransport,avmRemoteTransportRaw,attributeService,attributeServiceTransport,attributeServiceTransportRaw,jndiInfoBean,virtServerInfo,virtServerMBeanServer,virtServerRegistry,serverConnector,virtServerExporter,virtWebappRegistry,cacheControlInfo]; root of BeanFactory hierarchy]
03-Aug-2007 01:34:59 org.springframework.beans.factory.support.DefaultSingletonBeanRegistry destroySingletons
INFO: Destroying singletons in {org.springframework.beans.factory.support.DefaultListableBeanFactory defining beans [virtserver-properties,avmReauthenticatingAdvisor,reauthenticatingAdvice,clientTicketHolder,authenticationService,avmRemote,avmRemoteTransport,avmRemoteTransportRaw,attributeService,attributeServiceTransport,attributeServiceTransportRaw,jndiInfoBean,virtServerInfo,virtServerMBeanServer,virtServerRegistry,serverConnector,virtServerExporter,virtWebappRegistry,cacheControlInfo]; root of BeanFactory hierarchy}
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:615)
   at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:295)
   at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:433)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'avmReauthenticatingAdvisor' defined in URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver-context.xml]: Cannot resolve reference to bean 'reauthenticatingAdvice' while setting bean property 'advice'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'reauthenticatingAdvice' defined in URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver-context.xml]: Cannot resolve reference to bean 'authenticationService' while setting bean property 'authenticationService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authenticationService' defined in URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver-context.xml]: Invocation of init method failed; nested exception is org.springframework.remoting.RemoteLookupFailureException: Lookup of RMI stub failed; nested exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
   java.net.ConnectException: Connection refused
   at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:253)
   at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:124)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1019)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:809)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:425)
   at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:250)
   at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:141)
   at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:247)
   at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:161)
   at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:273)
   at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:346)
   at org.springframework.context.support.FileSystemXmlApplicationContext.<init>(FileSystemXmlApplicationContext.java:89)
   at org.springframework.context.support.FileSystemXmlApplicationContext.<init>(FileSystemXmlApplicationContext.java:74)
   at org.springframework.context.support.FileSystemXmlApplicationContext.<init>(FileSystemXmlApplicationContext.java:65)
   at org.alfresco.catalina.host.AVMHost.init(AVMHost.java:376)
   at org.alfresco.catalina.host.AVMHost.start(AVMHost.java:642)
   at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1014)
   at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
   at org.apache.catalina.core.StandardService.start(StandardService.java:448)
   at org.apache.catalina.core.StandardServer.start(StandardServer.java:700)
   at org.apache.catalina.startup.Catalina.start(Catalina.java:552)
   … 6 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'reauthenticatingAdvice' defined in URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver-context.xml]: Cannot resolve reference to bean 'authenticationService' while setting bean property 'authenticationService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authenticationService' defined in URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver-context.xml]: Invocation of init method failed; nested exception is org.springframework.remoting.RemoteLookupFailureException: Lookup of RMI stub failed; nested exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
   java.net.ConnectException: Connection refused
   at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:253)
   at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:124)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1019)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:809)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:425)
   at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:250)
   at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:141)
   at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:247)
   at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:161)
   at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:245)
   … 26 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authenticationService' defined in URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver-context.xml]: Invocation of init method failed; nested exception is org.springframework.remoting.RemoteLookupFailureException: Lookup of RMI stub failed; nested exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
   java.net.ConnectException: Connection refused
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1088)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:429)
   at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:250)
   at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:141)
   at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:247)
   at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:161)
   at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:245)
   … 35 more
Caused by: org.springframework.remoting.RemoteLookupFailureException: Lookup of RMI stub failed; nested exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
   java.net.ConnectException: Connection refused
   at org.springframework.remoting.rmi.RmiClientInterceptor.lookupStub(RmiClientInterceptor.java:214)
   at org.springframework.remoting.rmi.RmiClientInterceptor.prepare(RmiClientInterceptor.java:145)
   at org.springframework.remoting.rmi.RmiClientInterceptor.afterPropertiesSet(RmiClientInterceptor.java:133)
   at org.springframework.remoting.rmi.RmiProxyFactoryBean.afterPropertiesSet(RmiProxyFactoryBean.java:66)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1118)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1085)
   … 41 more
Caused by: java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
   java.net.ConnectException: Connection refused
   at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:590)
   at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:204)
   at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:190)
   at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:321)
   at sun.rmi.registry.RegistryImpl_Stub.lookup(RegistryImpl_Stub.java:88)
   at java.rmi.Naming.lookup(Naming.java:98)
   at org.springframework.remoting.rmi.RmiClientInterceptor.lookupStub(RmiClientInterceptor.java:199)
   … 46 more
Caused by: java.net.ConnectException: Connection refused
   at java.net.PlainSocketImpl.socketConnect(Native Method)
   at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:372)
   at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:233)
   at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:220)
   at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:385)
   at java.net.Socket.connect(Socket.java:541)
   at java.net.Socket.connect(Socket.java:491)
   at java.net.Socket.<init>(Socket.java:399)
   at java.net.Socket.<init>(Socket.java:208)
   at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:41)
   at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:145)
   at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:585)
   … 52 more

Outcomes