[#ABICLOUD-327] If you try to deploy a virtual appliance and a cloud node is not available, the deployment fails but no information about the reason is shown - ABIQUO

Abiquo

If you try to deploy a virtual appliance and a cloud node is not available, the deployment fails but no information about the reason is shown

Created: 13/Nov/09 06:14 PM Updated: 22/Feb/10 03:01 PM Resolved: 22/Feb/10 03:01 PM

Component/s:

Server, Virtualization

Affects Version/s:

abicloud-1.0.0-RC1

Fix Version/s:

abicloud-1.5.0-PR

Time Tracking:

Not Specified

Issue Links:

Related

This issue is related to:

~~ABICLOUD-385~~ All the WS-MAN plugins should implement a ''check'' using the GET operation.

Description

« Hide

If you deploy a virtual appliance but the cloud node fails, or the system runs out of cloud nodes, the deployment fails but no information about the reason is shown.

As a rule of thumb, before deploying a virtual appliance, the following checks and process should be performed:

1) Check that the chosen servers by the Schedulers are really up and running with the information stored in the system. If there are discrepances, then update the status in the database and reexecute the scheduling process and recheck the information obtained again.
2) If the platform does not have enough resources for the whole deployment, stop the deployment and rollback.
3) Reserve the resources temporaly during the deployment process, like a transaction.
4) Deploy the virtual images, and keep all the images suspended.
5) If there is a failure during the deployment, stop the deployment and rollback to status previous to step 1.
6) If the deployment is sucessful, change the status from suspended to running.
7) Confirm then that all the resources reserved temporaly are now permanent.

Also, when a failure happens during the deployment, a human readable error should be sent to the traces log AND the trace with the exception.

Sort Order:

[ Permalink | « Hide ]

Xavier Fernandez added a comment - 13/Nov/09 08:24 PM

Many point of this process is working now.

Pedro, please check it and add the points that now aren't

Thanks

[ Permalink | « Hide ]

Pedro Navarro Pérez added a comment - 17/Nov/09 01:06 PM

The steps have been checked, and the fist one is not implemented, so I've been opened a new ticket specific for this one [~~ABICLOUD-331~~]. The vmware plugin didn't propagate the error properly so it has been modified

[ Permalink | « Hide ]

Diego Parrilla added a comment - 30/Nov/09 10:36 PM

When the scheduler detects an issue when looking for new resources, it should report the user why it cannot allocate the resources. I have found these error when starting a new virtual app:

RECEIVED MESSAGE: Severity[CRITICAL], Event[VAPP_POWERON], Component[VIRTUAL_APP
LIANCE], Timestamp[1259634498876], Description[java.lang.Long cannot be cast to
java.math.BigDecimal], User[admin]
21:28:18.891 ERROR com.abiquo.util.AbiCloudError - Operation cannot be performed
: HibernateException has occurred while creating virtual machines[Error code:A
BI-S001 Timestamp:1259634498]. Operation cannot be performed:

Unable to create virtual machine(s) for the node(s) of the current virtual appli
ance
Error Code: ABI-S001
Error ID: 1259634498.
java.lang.ClassCastException: java.lang.Long cannot be cast to java.math.BigDeci
mal
        at com.abiquo.abiserver.persistence.dao.user.hibernate.EnterpriseDAOHibe
rnate.getTotalResourceUtilization(EnterpriseDAOHibernate.java:91) [EnterpriseDAO
Hibernate.class:na]
        at com.abiquo.abiserver.scheduler.ResourceAllocationLimitChecker.getAllR
esourceAllocationByEnterprise(ResourceAllocationLimitChecker.java:261) [Resource
AllocationLimitChecker.class:na]
        at com.abiquo.abiserver.scheduler.ResourceAllocationLimitChecker.checkRe
sourceAllocationLimit(ResourceAllocationLimitChecker.java:66) [ResourceAllocatio
nLimitChecker.class:na]
        at com.abiquo.abiserver.scheduler.SchedulerRestrictions.selectMachine(Sc
hedulerRestrictions.java:107) [SchedulerRestrictions.class:na]
        at com.abiquo.abiserver.commands.VirtualApplianceCommand.createNodeVirtu
alMachine(VirtualApplianceCommand.java:2041) [VirtualApplianceCommand.class:na]
        at com.abiquo.abiserver.commands.VirtualApplianceCommand.createVirtualMa
chines(VirtualApplianceCommand.java:2133) [VirtualApplianceCommand.class:na]
        at com.abiquo.abiserver.commands.VirtualApplianceCommand.startVirtualApp
liance(VirtualApplianceCommand.java:1615) [VirtualApplianceCommand.class:na]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [na:1.6.0
_17]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39) [na:1.6.0_17]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25) [na:1.6.0_17]
        at java.lang.reflect.Method.invoke(Method.java:597) [na:1.6.0_17]
        at com.abiquo.abiserver.commands.BasicCommand.onExecute(BasicCommand.jav
a:376) [BasicCommand.class:na]
        at com.abiquo.abiserver.commands.BasicCommand.execute(BasicCommand.java:
217) [BasicCommand.class:na]
        at com.abiquo.abiserver.commands.BasicCommand.execute(BasicCommand.java:
260) [BasicCommand.class:na]
        at com.abiquo.abiserver.services.flex.NonBlockingService.startVirtualApp
liance(NonBlockingService.java:166) [NonBlockingService.class:na]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [na:1.6.0
_17]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39) [na:1.6.0_17]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25) [na:1.6.0_17]
        at java.lang.reflect.Method.invoke(Method.java:597) [na:1.6.0_17]
        at flex.messaging.services.remoting.adapters.JavaAdapter.invoke(JavaAdap
ter.java:406) [flex-messaging-remoting-3.0.jar:3.0.0.544]
        at flex.messaging.services.RemotingService.serviceMessage(RemotingServic
e.java:183) [flex-messaging-remoting-3.0.jar:3.0.0.544]
        at flex.messaging.MessageBroker.routeMessageToService(MessageBroker.java
:1417) [flex-messaging-core-3.0.jar:3.0.0.544]
        at flex.messaging.endpoints.AbstractEndpoint.serviceMessage(AbstractEndp
oint.java:878) [flex-messaging-core-3.0.jar:3.0.0.544]
        at flex.messaging.endpoints.amf.MessageBrokerFilter.invoke(MessageBroker
Filter.java:121) [flex-messaging-core-3.0.jar:3.0.0.544]
        at flex.messaging.endpoints.amf.LegacyFilter.invoke(LegacyFilter.java:15
8) [flex-messaging-core-3.0.jar:3.0.0.544]
        at flex.messaging.endpoints.amf.SessionFilter.invoke(SessionFilter.java:
49) [flex-messaging-core-3.0.jar:3.0.0.544]
        at flex.messaging.endpoints.amf.BatchProcessFilter.invoke(BatchProcessFi
lter.java:67) [flex-messaging-core-3.0.jar:3.0.0.544]
        at flex.messaging.endpoints.amf.SerializationFilter.invoke(Serialization
Filter.java:146) [flex-messaging-core-3.0.jar:3.0.0.544]
        at flex.messaging.endpoints.BaseHTTPEndpoint.service(BaseHTTPEndpoint.ja
va:274) [flex-messaging-core-3.0.jar:3.0.0.544]
        at flex.messaging.MessageBrokerServlet.service(MessageBrokerServlet.java
:377) [flex-messaging-core-3.0.jar:3.0.0.544]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) [tomcat6
-servlet-2.5-api-6.0.18.jar:na]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appl
icationFilterChain.java:290) [catalina-6.0.18.jar:na]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationF
ilterChain.java:206) [catalina-6.0.18.jar:na]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperV
alve.java:233) [catalina-6.0.18.jar:na]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextV
alve.java:191) [catalina-6.0.18.jar:na]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.j
ava:128) [catalina-6.0.18.jar:na]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.j
ava:102) [catalina-6.0.18.jar:na]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineVal
ve.java:109) [catalina-6.0.18.jar:na]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.jav
a:286) [catalina-6.0.18.jar:na]
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java
:845) [tomcat-coyote-6.0.18.jar:na]
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.proce
ss(Http11Protocol.java:583) [tomcat-coyote-6.0.18.jar:na]
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:44
7) [tomcat-coyote-6.0.18.jar:na]
        at java.lang.Thread.run(Thread.java:619) [na:1.6.0_17]

[ Show » ]

Diego Parrilla added a comment - 30/Nov/09 10:36 PM When the scheduler detects an issue when looking for new resources, it should report the user why it cannot allocate the resources. I have found these error when starting a new virtual app: RECEIVED MESSAGE: Severity[CRITICAL], Event[VAPP_POWERON], Component[VIRTUAL_APP LIANCE], Timestamp[1259634498876], Description[java.lang.Long cannot be cast to java.math.BigDecimal], User[admin] 21:28:18.891 ERROR com.abiquo.util.AbiCloudError - Operation cannot be performed : HibernateException has occurred while creating virtual machines[Error code:A BI-S001 Timestamp:1259634498]. Operation cannot be performed: Unable to create virtual machine(s) for the node(s) of the current virtual appli ance Error Code: ABI-S001 Error ID: 1259634498. java.lang.ClassCastException: java.lang.Long cannot be cast to java.math.BigDeci mal at com.abiquo.abiserver.persistence.dao.user.hibernate.EnterpriseDAOHibe rnate.getTotalResourceUtilization(EnterpriseDAOHibernate.java:91) [EnterpriseDAO Hibernate.class:na] at com.abiquo.abiserver.scheduler.ResourceAllocationLimitChecker.getAllR esourceAllocationByEnterprise(ResourceAllocationLimitChecker.java:261) [Resource AllocationLimitChecker.class:na] at com.abiquo.abiserver.scheduler.ResourceAllocationLimitChecker.checkRe sourceAllocationLimit(ResourceAllocationLimitChecker.java:66) [ResourceAllocatio nLimitChecker.class:na] at com.abiquo.abiserver.scheduler.SchedulerRestrictions.selectMachine(Sc hedulerRestrictions.java:107) [SchedulerRestrictions.class:na] at com.abiquo.abiserver.commands.VirtualApplianceCommand.createNodeVirtu alMachine(VirtualApplianceCommand.java:2041) [VirtualApplianceCommand.class:na] at com.abiquo.abiserver.commands.VirtualApplianceCommand.createVirtualMa chines(VirtualApplianceCommand.java:2133) [VirtualApplianceCommand.class:na] at com.abiquo.abiserver.commands.VirtualApplianceCommand.startVirtualApp liance(VirtualApplianceCommand.java:1615) [VirtualApplianceCommand.class:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [na:1.6.0 _17] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:39) [na:1.6.0_17] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:25) [na:1.6.0_17] at java.lang.reflect.Method.invoke(Method.java:597) [na:1.6.0_17] at com.abiquo.abiserver.commands.BasicCommand.onExecute(BasicCommand.jav a:376) [BasicCommand.class:na] at com.abiquo.abiserver.commands.BasicCommand.execute(BasicCommand.java: 217) [BasicCommand.class:na] at com.abiquo.abiserver.commands.BasicCommand.execute(BasicCommand.java: 260) [BasicCommand.class:na] at com.abiquo.abiserver.services.flex.NonBlockingService.startVirtualApp liance(NonBlockingService.java:166) [NonBlockingService.class:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [na:1.6.0 _17] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:39) [na:1.6.0_17] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:25) [na:1.6.0_17] at java.lang.reflect.Method.invoke(Method.java:597) [na:1.6.0_17] at flex.messaging.services.remoting.adapters.JavaAdapter.invoke(JavaAdap ter.java:406) [flex-messaging-remoting-3.0.jar:3.0.0.544] at flex.messaging.services.RemotingService.serviceMessage(RemotingServic e.java:183) [flex-messaging-remoting-3.0.jar:3.0.0.544] at flex.messaging.MessageBroker.routeMessageToService(MessageBroker.java :1417) [flex-messaging-core-3.0.jar:3.0.0.544] at flex.messaging.endpoints.AbstractEndpoint.serviceMessage(AbstractEndp oint.java:878) [flex-messaging-core-3.0.jar:3.0.0.544] at flex.messaging.endpoints.amf.MessageBrokerFilter.invoke(MessageBroker Filter.java:121) [flex-messaging-core-3.0.jar:3.0.0.544] at flex.messaging.endpoints.amf.LegacyFilter.invoke(LegacyFilter.java:15 8) [flex-messaging-core-3.0.jar:3.0.0.544] at flex.messaging.endpoints.amf.SessionFilter.invoke(SessionFilter.java: 49) [flex-messaging-core-3.0.jar:3.0.0.544] at flex.messaging.endpoints.amf.BatchProcessFilter.invoke(BatchProcessFi lter.java:67) [flex-messaging-core-3.0.jar:3.0.0.544] at flex.messaging.endpoints.amf.SerializationFilter.invoke(Serialization Filter.java:146) [flex-messaging-core-3.0.jar:3.0.0.544] at flex.messaging.endpoints.BaseHTTPEndpoint.service(BaseHTTPEndpoint.ja va:274) [flex-messaging-core-3.0.jar:3.0.0.544] at flex.messaging.MessageBrokerServlet.service(MessageBrokerServlet.java :377) [flex-messaging-core-3.0.jar:3.0.0.544] at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) [tomcat6 -servlet-2.5-api-6.0.18.jar:na] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appl icationFilterChain.java:290) [catalina-6.0.18.jar:na] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationF ilterChain.java:206) [catalina-6.0.18.jar:na] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperV alve.java:233) [catalina-6.0.18.jar:na] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextV alve.java:191) [catalina-6.0.18.jar:na] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.j ava:128) [catalina-6.0.18.jar:na] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.j ava:102) [catalina-6.0.18.jar:na] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineVal ve.java:109) [catalina-6.0.18.jar:na] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.jav a:286) [catalina-6.0.18.jar:na] at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java :845) [tomcat-coyote-6.0.18.jar:na] at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.proce ss(Http11Protocol.java:583) [tomcat-coyote-6.0.18.jar:na] at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:44 7) [tomcat-coyote-6.0.18.jar:na] at java.lang.Thread.run(Thread.java:619) [na:1.6.0_17]

[ Permalink | « Hide ]

Diego Parrilla added a comment - 30/Nov/09 10:38 PM - edited

Modified to 'blocker'. System become unstable and error cannot be recovered.

[ Permalink | « Hide ]

Pedro Navarro Pérez added a comment - 01/Dec/09 04:57 PM

- Please take a look to the latest comment

[ Permalink | « Hide ]

Albert Puig added a comment - 01/Dec/09 05:38 PM

resolved ClassCastException.

IMO this fix do not close the ''bug'', there is much more work to do in order to assure physical machine availability (on the roadmap).

should i reasing the the bug ? decrease the severity ?

[ Permalink | « Hide ]

Xavier Fernandez added a comment - 01/Dec/09 05:47 PM

Changed severity. But we need to close this issue until we support

1) Check that the chosen servers by the Schedulers are really up and running with the information stored in the system. If there are discrepances, then update the status in the database and reexecute the scheduling process and recheck the information obtained again.