Search code, repositories, users, issues, pull requests...

@boring-cyborg boring-cyborg bot added component:api component:kvm labels

Nov 24, 2025

@pavanaravapalli pavanaravapalli marked this pull request as draft

November 24, 2025 22:22

@pavanaravapalli pavanaravapalli changed the title ~~(削除) Fix: clean up CloudStack metadata (zone, pod, cluster, host details) for vm (kvm) unmanage operation. (削除ここまで)~~ (追記) WIP: clean up CloudStack metadata (zone, pod, cluster, host details) for vm (kvm) unmanage operation. (追記ここまで)

Nov 24, 2025

@DaanHoogland DaanHoogland added this to the 4.23 milestone

Nov 25, 2025

@codecov

Copy link

codecov bot commented Nov 25, 2025 •

edited

Loading

Codecov Report

❌ Patch coverage is 53.84615% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.57%. Comparing base (8171d95) to head (6632809).
⚠️ Report is 91 commits behind head on main.

Files with missing lines	Patch %	Lines
...a/com/cloud/agent/api/UnmanageInstanceCommand.java	50.00%	2 Missing ⚠️
...wrapper/LibvirtUnmanageInstanceCommandWrapper.java	0.00%	2 Missing ⚠️
.../java/com/cloud/agent/api/to/VirtualMachineTO.java	50.00%	1 Missing ⚠️
...ervisor/kvm/resource/LibvirtComputingResource.java	50.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@ Coverage Diff @@
## main #12127 +/- ##
=========================================
 Coverage 17.56% 17.57% 
- Complexity 15538 15547 +9 
=========================================
 Files 5912 5913 +1 
 Lines 529383 529452 +69 
 Branches 64660 64671 +11 
=========================================
+ Hits 92984 93026 +42 
- Misses 425941 425967 +26 
- Partials 10458 10459 +1

Flag	Coverage Δ
uitests	`3.58% <ø> (+<0.01%)`	⬆️
unittests	`18.63% <53.84%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

Contributor

DaanHoogland commented Nov 25, 2025

@pavanaravapalli , looks good. Please forgive me if I am mistaken, but didn’t we agree that leaving metadata in unmanaged VMs should be optional?
I see you unconditionally remove it on un-manage.

Copy link

Contributor

DaanHoogland commented Nov 25, 2025

@blueorangutan package

Copy link

blueorangutan commented Nov 25, 2025

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Copy link

blueorangutan commented Nov 25, 2025

Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 15830

Copy link

Contributor Author

pavanaravapalli commented Nov 25, 2025 •

edited

Loading

Thanks for checking. I wasn’t fully aligned with leaving the metadata as-is without a cleanup option, since it can lead to repetitive manual work and also raises data-security concerns when unmanaged VMs keep CS details.

As this PR is still in progress, I plan to add a toggle kvm.libvirt.xml.metadata.clean=true/false( global config ) to control metadata cleanup on unmanage.


 Added support to remove CloudStack metadata (zone, pod, cluster, host...

... details) from Libvirt XML for KVM VMs after an unmanage operation, controlled by the global config vm.unmanage.libvirt.metadata.cleanup [default: true]

@pavanaravapalli pavanaravapalli force-pushed the remove-libvirt-metadata-unmanaged-kvm-vm branch from 4ed2729 to 6632809 Compare

November 27, 2025 08:32

@boring-cyborg boring-cyborg bot added component:compute component:orchestration labels

Copy link

Contributor Author

pavanaravapalli commented Nov 27, 2025

@blueorangutan package

Copy link

blueorangutan commented Nov 27, 2025

@pavanaravapalli a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@pavanaravapalli pavanaravapalli changed the title ~~(削除) WIP: clean up CloudStack metadata (zone, pod, cluster, host details) for vm (kvm) unmanage operation. (削除ここまで)~~ (追記) clean up CloudStack metadata (zone, pod, cluster, host details) for vm (kvm) unmanage operation. (追記ここまで)

Copy link

blueorangutan commented Nov 27, 2025

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15840

@pavanaravapalli pavanaravapalli marked this pull request as ready for review

November 27, 2025 12:22

DaanHoogland

DaanHoogland approved these changes

View reviewed changes

Copy link

Contributor

@DaanHoogland DaanHoogland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clgtm

@DaanHoogland DaanHoogland added the status:needs-testing label

Copy link

Contributor

DaanHoogland commented Nov 27, 2025

@blueorangutan test

Copy link

blueorangutan commented Nov 27, 2025

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

Copy link

blueorangutan commented Nov 27, 2025

[SF] Trillian Build Failed (tid-14876)

Copy link

Contributor Author

pavanaravapalli commented Nov 27, 2025

I believe the Jenkins build issues aren’t related to my code changes; they seem to be caused by the DB schema upgrade.

Copy link

Contributor

DaanHoogland commented Nov 27, 2025

@blueorangutan package

Copy link

blueorangutan commented Nov 27, 2025

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Copy link

blueorangutan commented Nov 27, 2025

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✖️ debian ✔️ suse15. SL-JID 15844

Copy link

Contributor

DaanHoogland commented Nov 28, 2025

@blueorangutan test

Copy link

blueorangutan commented Nov 28, 2025

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

Copy link

blueorangutan commented Nov 29, 2025

[SF] Trillian test result (tid-14879)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 51927 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12127-t14879-kvm-ol8.zip
Smoke tests completed. 149 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_unmanage_vm_cycle_persistent_domain	`Error`	68.43	test_vm_lifecycle_unmanage_kvm_import.py
test_01_unmanage_vm_cycle_persistent_domain	`Error`	68.44	test_vm_lifecycle_unmanage_kvm_import.py
ContextSuite context=TestUnmanageKvmVM>:teardown	`Error`	240.96	test_vm_lifecycle_unmanage_kvm_import.py

@DaanHoogland DaanHoogland closed this

Dec 1, 2025

@DaanHoogland DaanHoogland reopened this

Dec 1, 2025

@DaanHoogland DaanHoogland requested review from RosiKyu, bernardodemarco and nvazquez

December 1, 2025 07:12

@RosiKyu RosiKyu assigned RosiKyu and unassigned RosiKyu

Jan 26, 2026

@vladimirpetrov vladimirpetrov self-assigned this

Jan 28, 2026

@vladimirpetrov

Copy link

Contributor

vladimirpetrov commented Jan 28, 2026

@blueorangutan package

Copy link

blueorangutan commented Jan 28, 2026

@vladimirpetrov a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

Copy link

blueorangutan commented Jan 28, 2026

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16583

@vladimirpetrov

Copy link

Contributor

vladimirpetrov commented Jan 30, 2026

I'm getting the following error when trying to unmanage instance @pavanaravapalli:

image

2026年01月30日 16:31:41,120 DEBUG [c.c.a.t.Request] (API-Job-Executor-39:[ctx-c0e7c82c, job-48, ctx-89088ece]) (logid:64a7100b) Seq 1-5360972406431160699: Sending { Cmd , MgmtId: 32988536504755, via: 1(ref-tr
l-10755-k-Mol9-vladimir-petrov-kvm1), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.UnmanageInstanceCommand":{"instanceName":"i-2-5-VM","executeInSequence":"false","isConfigDriveAttached":"false","isLibvi
rtMetadataCleanUpEnabled":"true","wait":"0","bypassHostMaintenance":"false"}}] }
2026年01月30日 16:31:41,164 DEBUG [c.c.a.t.Request] (AgentManager-Handler-15:[]) (logid:) Seq 1-5360972406431160699: Processing: { Ans: , MgmtId: 32988536504755, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent
.api.Answer":{"result":"false","details":"java.lang.NullPointerException: Cannot invoke "com.cloud.agent.api.to.VirtualMachineTO.setExcludeMetadata(boolean)" because "vmSpec" is null
 at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtUnmanageInstanceCommandWrapper.execute(LibvirtUnmanageInstanceCommandWrapper.java:71)
 at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtUnmanageInstanceCommandWrapper.execute(LibvirtUnmanageInstanceCommandWrapper.java:62)
 at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
 at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:2282)
 at com.cloud.agent.Agent.processRequest(Agent.java:813)
 at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1295)
 at com.cloud.utils.nio.Task.call(Task.java:83)
 at com.cloud.utils.nio.Task.call(Task.java:29)
 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
 at java.base/java.lang.Thread.run(Thread.java:1583)
","wait":"0","bypassHostMaintenance":"false"}}] }
2026年01月30日 16:31:41,164 DEBUG [c.c.a.t.Request] (API-Job-Executor-39:[ctx-c0e7c82c, job-48, ctx-89088ece]) (logid:64a7100b) Seq 1-5360972406431160699: Received: { Ans: , MgmtId: 32988536504755, via: 1(ref
-trl-10755-k-Mol9-vladimir-petrov-kvm1), Ver: v1, Flags: 10, { Answer } }
2026年01月30日 16:31:41,164 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (API-Job-Executor-39:[ctx-c0e7c82c, job-48, ctx-89088ece]) (logid:64a7100b) Failed to persist domain XML for Instance: i-2-5-VM on h
ost ID: 1
2026年01月30日 16:31:41,165 ERROR [c.c.v.UserVmManagerImpl] (API-Job-Executor-39:[ctx-c0e7c82c, job-48, ctx-89088ece]) (logid:64a7100b) Could not unmanage VM VM instance {"id":5,"instanceName":"i-2-5-VM","stat
e":"Running","type":"User","uuid":"b4ad6cd5-68b2-47ea-9044-303c39cf7148"} com.cloud.utils.exception.CloudRuntimeException: Failed to persist domain XML for Instance: i-2-5-VM on host ID: 1
 at com.cloud.vm.VirtualMachineManagerImpl.persistDomainForKVM(VirtualMachineManagerImpl.java:2103)
 at com.cloud.vm.VirtualMachineManagerImpl.unmanage(VirtualMachineManagerImpl.java:2037)
 at com.cloud.vm.UserVmManagerImpl.unmanageUserVM(UserVmManagerImpl.java:9541)
 at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
 at java.base/java.lang.reflect.Method.invoke(Method.java:580)
 at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
 at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
 at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
 at jdk.proxy3/jdk.proxy3.$Proxy221.unmanageUserVM(Unknown Source)
 at org.apache.cloudstack.vm.UnmanagedVMsManagerImpl.unmanageVMInstance(UnmanagedVMsManagerImpl.java:2466)
 at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
 at java.base/java.lang.reflect.Method.invoke(Method.java:580)
 at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
 at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:109)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
 at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
 at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
 at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
 at jdk.proxy3/jdk.proxy3.$Proxy541.unmanageVMInstance(Unknown Source)
 at org.apache.cloudstack.api.command.admin.vm.UnmanageVMInstanceCmd.execute(UnmanageVMInstanceCmd.java:125)
 at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:173)
 at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:110)
 at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl5ドル.runInContext(AsyncJobManagerImpl.java:698)
 at org.apache.cloudstack.managed.context.ManagedContextRunnable1ドル.run(ManagedContextRunnable.java:49)
 at org.apache.cloudstack.managed.context.impl.DefaultManagedContext1ドル.call(DefaultManagedContext.java:56)
 at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
 at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
 at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
 at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl5ドル.run(AsyncJobManagerImpl.java:646)
 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
 at java.base/java.lang.Thread.run(Thread.java:1583)

when the global setting vm.unmanage.libvirt.metadata.cleanup = true. When it's set to 'false', the operation is successful.

Copy link

Contributor Author

pavanaravapalli commented Jan 31, 2026

Okay i will check it.