vDirector 5.1 sql upgrade failing

Yesterday we saw that not to much people upgraded their vCloud environment to 5.1

We hanged on step 2.0.34. it has a very funny when you heard that the problem , Problem is mem_overhead column.
At the upgrade time sql upgrade script create some temporary tables and this temp tables columns type of  mem_overhead is “int” but real world active running persistent tables mem_overhead is bigint and this cause vDirector upgrade problem …

Solution

First, restore the old db , select the values from vm_in table and find out something more then nine digits

select * from vm_inv where mem_overhead=3968430080

Replace it with any nine digits like below

update  vm_inv set mem_overhead=396843008  where mem_overhead=3968430080

VMware Support answer us like below 

I just got confirmation from engineering team that the memory_overhead_mb column in computevm table, it is deleted later in the upgrade script. So we don’t need to worry about about if this int type will cause any further issue with vCD5.1.
The workaround we discussed( just change the int to bigint), it should also work.And since the column is deleted at the end of upgrade, so we also don’t need to worry the type change has some downside in the future.

Hope no body need this article, but if you needed then you have a solution.

Note to all, always do tests with real data on test environment maybe still you can not see something but still you have chance to catch it !

VM

Full Error is Below ;

2012-10-01 19:34:06,380 | INFO | main | UpgradeAgent | Executing upgrade task: Upgrade to 2.0.34 |
2012-10-01 19:34:06,381 | DEBUG | pool-1-thread-1 | LoggingProgressListener | Upgrade Task Received progress report: com.vmware.vcloud.progress.ImmutableProgressReport@65f4be12 {state: RUNNING, progress: 0%} |
2012-10-01 19:34:06,383 | DEBUG | pool-1-thread-1 | LoggingProgressListener | Upgrade Task Received progress report: com.vmware.vcloud.progress.ImmutableProgressReport@65f4bfa5 {state: RUNNING, progress: 13%} |
2012-10-01 19:34:06,384 | DEBUG | pool-1-thread-1 | SerialAggregateTask | Steps to upgrade to 2.0.34: Beginning execution of task Record version 2.0.33.transition |
2012-10-01 19:34:06,427 | DEBUG | pool-1-thread-1 | SerialAggregateTask | Steps to upgrade to 2.0.34: Beginning execution of task Compute.groovy:22 |
2012-10-01 19:34:06,427 | DEBUG | pool-1-thread-1 | RawSQLTask | Executing sql ‘INSERT INTO computevm (id, computehub_id, vrp_id, creation_status, memory_min_mb,
memory_configured_mb, memory_overhead_mb, cpu_min_mhz, num_vcpu, vmmoref)
SELECT distinct vm.id, vrprp.computehub_id, vrp.id,
0,
vminv.mem_reservation, vminv.mem, vminv.mem_overhead,
vminv.cpu_reservation, vminv.vcpu_count,
vm.moref
FROM vm vm, vm_inv vminv, vapp_vm vappvm,
vrp vrp, vrp_rp vrprp,
vm_container vmcont,
org_prov_vdc o, org_prov_vdc_rp orp
WHERE vm.moref = vminv.moref
AND vm.vc_id = vminv.vc_id
AND vm.id NOT IN (SELECT vm_id FROM deployed_vm)
AND vm.id = vappvm.svm_id
AND vappvm.vapp_id = vmcont.sg_id
AND vmcont.org_vdc_id = o.id
AND o.id = orp.org_prov_vdc_id
AND orp.rp_moref = vrprp.sub_rp_moref
AND orp.vc_id=vrprp.sub_rp_vc_id
AND vrp.id = vrprp.vrp_id
AND vminv.resource_pool_moref = vrprp.sub_rp_moref’ |
2012-10-01 19:34:06,861 | WARN | pool-1-thread-1 | SerialAggregateTask | Steps to upgrade to 2.0.34: Task failed due to uncaught exception |
java.sql.DataTruncation: Data truncation
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:382)
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2816)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2254)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:631)
at net.sourceforge.jtds.jdbc.JtdsStatement.processResults(JtdsStatement.java:584)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL(JtdsStatement.java:546)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeImpl(JtdsStatement.java:723)
at net.sourceforge.jtds.jdbc.JtdsStatement.execute(JtdsStatement.java:1157)
at com.vmware.vcloud.upgrade.tasks.RawSQLTask.call(RawSQLTask.java:106)
at com.vmware.vcloud.upgrade.tasks.RawSQLTask.call(RawSQLTask.java:1)
at com.vmware.vcloud.upgrade.SerialAggregateTask.call(SerialAggregateTask.java:43)
at com.vmware.vcloud.upgrade.SerialAggregateTask.call(SerialAggregateTask.java:1)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.doCall(AbstractDelegatingTask.java:124)
at com.vmware.vcloud.upgrade.TransactionTask.doCall(TransactionTask.java:74)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.call(AbstractDelegatingTask.java:103)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.call(AbstractDelegatingTask.java:1)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.doCall(AbstractDelegatingTask.java:124)

at com.vmware.vcloud.upgrade.UpgradeTask.doCall(UpgradeTask.java:79)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.call(AbstractDelegatingTask.java:103)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.call(AbstractDelegatingTask.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2012-10-01 19:34:06,862 | DEBUG | pool-1-thread-1 | RawSQLTask | Transaction for task ‘com.vmware.vcloud.upgrade.SerialAggregateTask@3a97263f’ will rollback |
java.sql.DataTruncation: Data truncation
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:382)
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2816)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2254)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:631)
at net.sourceforge.jtds.jdbc.JtdsStatement.processResults(JtdsStatement.java:584)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL(JtdsStatement.java:546)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeImpl(JtdsStatement.java:723)
at net.sourceforge.jtds.jdbc.JtdsStatement.execute(JtdsStatement.java:1157)
at com.vmware.vcloud.upgrade.tasks.RawSQLTask.call(RawSQLTask.java:106)
at com.vmware.vcloud.upgrade.tasks.RawSQLTask.call(RawSQLTask.java:1)
at com.vmware.vcloud.upgrade.SerialAggregateTask.call(SerialAggregateTask.java:43)
at com.vmware.vcloud.upgrade.SerialAggregateTask.call(SerialAggregateTask.java:1)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.doCall(AbstractDelegatingTask.java:124)
at com.vmware.vcloud.upgrade.TransactionTask.doCall(TransactionTask.java:74)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.call(AbstractDelegatingTask.java:103)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.call(AbstractDelegatingTask.java:1)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.doCall(AbstractDelegatingTask.java:124)
at com.vmware.vcloud.upgrade.UpgradeTask.doCall(UpgradeTask.java:79)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.call(AbstractDelegatingTask.java:103)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.call(AbstractDelegatingTask.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2012-10-01 19:34:06,864 | DEBUG | pool-1-thread-1 | LoggingProgressListener | Upgrade Task Received progress report: com.vmware.vcloud.progress.ImmutableProgressReport@702d32f8 {state: FAILED, progress: 13%} |
2012-10-01 19:34:06,866 | ERROR | main | UpgradeAgent | Unable to upgrade the database: java.sql.DataTruncation: Data truncation |
com.vmware.vcloud.upgrade.UpgradeAgentException: java.sql.DataTruncation: Data truncation
at com.vmware.vcloud.upgrade.UpgradeAgent.runUpgradeTasks(UpgradeAgent.java:1219)
at com.vmware.vcloud.upgrade.UpgradeAgent.executeUpgrade(UpgradeAgent.java:745)
at com.vmware.vcloud.upgrade.UpgradeAgent.start(UpgradeAgent.java:979)
at com.vmware.vcloud.upgrade.UpgradeAgent.main(UpgradeAgent.java:881)
Caused by: java.sql.DataTruncation: Data truncation
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:382)
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2816)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2254)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:631)
at net.sourceforge.jtds.jdbc.JtdsStatement.processResults(JtdsStatement.java:584

at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL(JtdsStatement.java:546)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeImpl(JtdsStatement.java:723)
at net.sourceforge.jtds.jdbc.JtdsStatement.execute(JtdsStatement.java:1157)
at com.vmware.vcloud.upgrade.tasks.RawSQLTask.call(RawSQLTask.java:106)
at com.vmware.vcloud.upgrade.tasks.RawSQLTask.call(RawSQLTask.java:1)
at com.vmware.vcloud.upgrade.SerialAggregateTask.call(SerialAggregateTask.java:43)
at com.vmware.vcloud.upgrade.SerialAggregateTask.call(SerialAggregateTask.java:1)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.doCall(AbstractDelegatingTask.java:124)
at com.vmware.vcloud.upgrade.TransactionTask.doCall(TransactionTask.java:74)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.call(AbstractDelegatingTask.java:103)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.call(AbstractDelegatingTask.java:1)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.doCall(AbstractDelegatingTask.java:124)
at com.vmware.vcloud.upgrade.UpgradeTask.doCall(UpgradeTask.java:79)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.call(AbstractDelegatingTask.java:103)
at com.vmware.vcloud.upgrade.AbstractDelegatingTask.call(AbstractDelegatingTask.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

Posted on 02/10/2012, in vDirector 5.1 and tagged , , , , , , , , , , . Bookmark the permalink. 1 Comment.

  1. Sascha Kloeppel

    Your’re my super hero. This made the trick. Nasty litte 10 digit entries.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: