CLOUDSTACK-10290: Introduce option to have config drives on primary storage by yadvr · Pull Request #2651 · apache/cloudstack

yadvr · 2018-05-17T15:38:17Z

This introduces a new global setting vm.configdrive.primarypool.enabled to toggle creation/hosting of config drive iso files on primary storage, the default will be false causing them to be hosted on secondary storage. The support is limited from hypervisor resource side and in current implementation limited to KVM. The next big change is that config drive is created at a temporary location by management server and shipped to either KVM or SSVM agent via cmd-answer pattern, the data of which is not logged in logs. This saves us from adding genisoimage dependency on cloudstack-agent pkg.

The APIs to reset ssh public key, password and user-data (via update VM API) requires that VM should be shutdown. Therefore, in the refactoring I removed the case of updation of existing ISO. If there are objections I'll re-put the strategy to detach+attach new config iso as a way of updation. In the refactored implementation, the folder name is changed to lower-cased configdrive. And during VM start, migration or shutdown/removal if primary storage is enable for use, the KVM agent will handle cleanup tasks otherwise SSVM agent will handle them.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

GitHub Issue/PRs

Screenshots (if appropriate):

How Has This Been Tested?

Checklist:

I have read the CONTRIBUTING document.
My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
Testing
I have added tests to cover my changes.
All relevant new and existing integration tests have passed.
A full integration testsuite with all test that can run on my environment has passed.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

…api is called when VM was running Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

…c key Refactor code Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

yadvr · 2018-05-17T15:39:03Z

Work left is some testing around VM migration and unit test/marvin test fixing.
@blueorangutan package

blueorangutan · 2018-05-17T15:39:52Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

blueorangutan · 2018-05-17T16:09:16Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2061

yadvr · 2018-05-17T16:27:47Z

@blueorangutan test matrix

blueorangutan · 2018-05-17T16:28:52Z

@rhtyd a Trillian-Jenkins matrix job (centos6 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

blueorangutan · 2018-05-17T23:12:01Z

Trillian test result (tid-2682)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 22215 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2651-t2682-kvm-centos7.zip
Intermitten failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermitten failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 64 look OK, 3 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestDeployVirtioSCSIVM>:setup	`Error`	0.00	test_deploy_virtio_scsi_vm.py
test_03_vpc_privategw_restart_vpc_cleanup	`Error`	106.11	test_privategw_acl.py
test_hostha_enable_ha_when_host_in_maintenance	`Error`	4.60	test_hostha_kvm.py

blueorangutan · 2018-05-18T00:04:48Z

Trillian test result (tid-2683)
Environment: vmware-65 (x2), Advanced Networking with Mgmt server 7
Total time taken: 25555 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2651-t2683-vmware-65.zip
Intermitten failure detected: /marvin/tests/smoke/test_deploy_vm_root_resize.py
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Smoke tests completed. 66 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_03_vpc_privategw_restart_vpc_cleanup	`Error`	212.31	test_privategw_acl.py

borisstoyanov · 2018-05-18T07:30:16Z

@blueorangutan test centos7 xenserver-71

blueorangutan · 2018-05-18T07:30:51Z

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + xenserver-71) has been kicked to run smoke tests

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

yadvr · 2018-05-18T11:53:35Z

@blueorangutan package

blueorangutan · 2018-05-18T11:53:51Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

nvazquez · 2018-05-18T12:13:44Z

...ne/storage/configdrive/src/org/apache/cloudstack/storage/configdrive/ConfigDriveBuilder.java

+            }
+
+            JsonObject metaData = new JsonObject();
+            for (String[] item : vmData) {


Can this logic in the FOR loop be extracted to a method?

To limit my effort scope, code in this class were previously in Nfs resource class. Most of the changes in this class are from a move-refactor. I'll further refactor if I've time, so yes I can try.

nvazquez · 2018-05-18T12:15:42Z

...ne/storage/configdrive/src/org/apache/cloudstack/storage/configdrive/ConfigDriveBuilder.java

+            }
+
+            File tmpIsoStore = new File(tempDirName, new File(isoFileName).getName());
+            Script command = new Script("/usr/bin/genisoimage", Duration.standardSeconds(300), LOG);


Can this also be extracted into a method?

nvazquez

Code LGTM, just left some minor comments

blueorangutan · 2018-05-18T12:21:25Z

Packaging result: ✔centos6 ✔centos7 ✖debian. JID-2065

yadvr · 2018-05-18T12:22:09Z

@blueorangutan test

blueorangutan · 2018-05-18T12:22:52Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2018-05-18T14:42:24Z

Trillian test result (tid-2685)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 24381 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2651-t2685-xenserver-71.zip
Intermitten failure detected: /marvin/tests/smoke/test_deploy_vm_iso.py
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Smoke tests completed. 66 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_03_vpc_privategw_restart_vpc_cleanup	`Error`	195.54	test_privategw_acl.py

blueorangutan · 2018-05-19T01:26:32Z

Trillian test result (tid-2689)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45321 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2651-t2689-kvm-centos7.zip
Intermitten failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermitten failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermitten failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 62 look OK, 5 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestDeployVirtioSCSIVM>:setup	`Error`	0.00	test_deploy_virtio_scsi_vm.py
test_03_vpc_privategw_restart_vpc_cleanup	`Failure`	267.73	test_privategw_acl.py
test_04_rvpc_privategw_static_routes	`Failure`	430.96	test_privategw_acl.py
test_02_not_secured_vm_migration	`Error`	82.78	test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration	`Error`	7.21	test_vm_life_cycle.py
test_04_nonsecured_to_secured_vm_migration	`Error`	56.18	test_vm_life_cycle.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Failure`	541.73	test_vpc_redundant.py
test_02_redundant_VPC_default_routes	`Failure`	359.32	test_vpc_redundant.py
test_02_VPC_default_routes	`Failure`	283.54	test_vpc_router_nics.py

yadvr · 2018-05-19T05:09:38Z

@blueorangutan test

blueorangutan · 2018-05-19T05:09:52Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2018-05-19T19:50:23Z

Trillian test result (tid-2690)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 51048 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2651-t2690-kvm-centos7.zip
Intermitten failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermitten failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermitten failure detected: /marvin/tests/smoke/test_ssvm.py
Intermitten failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermitten failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 63 look OK, 4 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestDeployVirtioSCSIVM>:setup	`Error`	0.00	test_deploy_virtio_scsi_vm.py
test_03_vpc_privategw_restart_vpc_cleanup	`Failure`	287.27	test_privategw_acl.py
test_04_rvpc_privategw_static_routes	`Failure`	521.53	test_privategw_acl.py
test_03_secured_to_nonsecured_vm_migration	`Error`	11.40	test_vm_life_cycle.py
test_10_attachAndDetach_iso	`Error`	28.31	test_vm_life_cycle.py
test_hostha_enable_ha_when_host_in_maintenance	`Error`	4.65	test_hostha_kvm.py

borisstoyanov

I've just run a test and can confirm it uses the primary storage to store the config drive iso, also extracted the iso and open it and it shows the data I've put it, LGTM. Turning the feature on and off also moves the iso to the s/p storage.

yadvr · 2018-05-21T06:53:13Z

Based on code reviews and both manual and regression tests. I'll merge this. Further refactorings can be done in a future PR.

borisstoyanov

I've just did a manual testing on isolated, shared and l2 types of networks. Config drive iso appears on primary containing all the user data configured. I've also performed some lifecycle operations like start/stop, reboot, migrate, reinstall and it works fine. One note is that if the user wants to update userdata on running VM it needs to stop the VM, execute the update userdata API and start the VM. LGTM

yadvr · 2018-05-21T08:56:20Z

Merging based on code reviews, test results incl. manual tests.

…2651) Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

yadvr added 7 commits May 17, 2018 01:15

cfgdrive: introduce new engine storage component and add rpm deps

ee4f6cf

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

cfg: introduce cfg storage component deps to ssvm

f7b725a

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

refactors, save metadata on update/reset APIs before rebooting VM is …

3247ad5

…api is called when VM was running Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

reset pass should fail if VM is not stopped just like reset ssh publi…

3db3271

…c key Refactor code Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

cfg: build iso on mgmt server and ship iso-data around

184de36

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

implement config drive on primary storage for KVM + global setting

4d4dd02

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

minor fixes, don't log iso data etc.

0b7aac5

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

yadvr added this to the 4.11.1.0 milestone May 17, 2018

yadvr self-assigned this May 17, 2018

yadvr requested review from DaanHoogland and fmaximus May 17, 2018 15:38

yadvr added the type:config label May 17, 2018

yadvr requested a review from rafaelweingartner May 17, 2018 15:38

remove deps

8bb8e0a

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

apache deleted a comment from blueorangutan May 18, 2018

yadvr added 2 commits May 18, 2018 16:26

fix vm migration with config drive on primary storage

40ace72

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

cleanup and fix tests

57f2dfd

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

yadvr added component:kvm component:primary-storage component:secondary-storage type:cleanup labels May 18, 2018

yadvr mentioned this pull request May 18, 2018

WIP perform config drive creation on primary storage (CLOUDSTACK-10290) #2614

Closed

12 tasks

yadvr requested review from PaulAngus and nvazquez May 18, 2018 11:47

nvazquez reviewed May 18, 2018

View reviewed changes

nvazquez approved these changes May 18, 2018

View reviewed changes

borisstoyanov approved these changes May 20, 2018

View reviewed changes

borisstoyanov approved these changes May 21, 2018

View reviewed changes

Merge branch '4.11' into cfgdrv-on-primary

173c611

yadvr merged commit acc5fdc into apache:4.11 May 21, 2018

yadvr added a commit that referenced this pull request May 21, 2018

Merge branch '4.11': allow config drives on primary storage for KVM (#…

7c6777b

…2651) Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

DaanHoogland mentioned this pull request May 24, 2018

configdrive testing needs refactorring #2671

Closed

fmaximus mentioned this pull request May 31, 2018

Fix test_configdrive.py and test_nuage_configdrive #2673

Merged

12 tasks

Conversation

yadvr commented May 17, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Types of changes

GitHub Issue/PRs

Screenshots (if appropriate):

How Has This Been Tested?

Checklist:

Uh oh!

yadvr commented May 17, 2018

Uh oh!

blueorangutan commented May 17, 2018

Uh oh!

blueorangutan commented May 17, 2018

Uh oh!

yadvr commented May 17, 2018

Uh oh!

blueorangutan commented May 17, 2018

Uh oh!

blueorangutan commented May 17, 2018

Uh oh!

blueorangutan commented May 18, 2018

Uh oh!

borisstoyanov commented May 18, 2018

Uh oh!

blueorangutan commented May 18, 2018

Uh oh!

yadvr commented May 18, 2018

Uh oh!

blueorangutan commented May 18, 2018

Uh oh!

nvazquez May 18, 2018

Choose a reason for hiding this comment

Uh oh!

yadvr May 18, 2018

Choose a reason for hiding this comment

Uh oh!

nvazquez May 18, 2018

Choose a reason for hiding this comment

Uh oh!

nvazquez left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented May 18, 2018

Uh oh!

yadvr commented May 18, 2018

Uh oh!

blueorangutan commented May 18, 2018

Uh oh!

blueorangutan commented May 18, 2018

Uh oh!

blueorangutan commented May 19, 2018

Uh oh!

yadvr commented May 19, 2018

Uh oh!

blueorangutan commented May 19, 2018

Uh oh!

blueorangutan commented May 19, 2018

Uh oh!

borisstoyanov left a comment

Choose a reason for hiding this comment

Uh oh!

yadvr commented May 21, 2018

Uh oh!

borisstoyanov left a comment

Choose a reason for hiding this comment

Uh oh!

yadvr commented May 21, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

yadvr commented May 17, 2018 •

edited

Loading