You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
(docs) VM.migrate.md: Rephrase and simplify, improve readability (#6307)
(docs) Update the walk-trough `VM.migrate`:
- Include the Live migration flowchart for reference and as overview.
- Fix the chapter structure and chapter headings: Improved table of
contents (1st button in the top bar)
- Add chapter links to the mentioned atomics operations
- Covert a long sentence describing a list of parameters into a simple
list of parameters
- Clarify ambiguities, e.g. "if we are already at the right place" ->
"if the command is already at the destination host"
- Removed the use of the word "will" when things are already happening
or are done just at that point.
- Removed sentences that just filled space by just mentioning what the
next chapter headline says.
- Improved the chapter on the final step with more references and a link
to a helpful explanation.
- Updated the links for a better understanding to the currently used
URLs.
that has low level operations performed by the backend. These atomics operations
19
-
that we will describe in the documentation are:
20
-
21
-
- VM.restore
22
-
- VM.rename
23
-
- VBD.set_active
24
-
- VBD.plug
25
-
- VIF.set_active
26
-
- VGPU.set_active
27
-
- VM.create_device_model
28
-
- PCI.plug
29
-
- VM.set_domain_action_request
30
-
31
-
The command has several parameters such as: Should it be started asynchronously,
32
-
should it be forwarded to another host, how arguments should be marshalled and
33
-
so on. A new thread is created by [xapi/server_helpers.ml](https://github.com/xapi-project/xen-api/blob/7ac88b90e762065c5ebb94a8ea61c61bdbf62c5c/ocaml/xapi/server_helpers.ml#L55)
34
-
to handle the command asynchronously. At this point the helper also check if
20
+
that uses many low level atomics operations. These are:
The migrate command has several parameters such as:
32
+
33
+
- Should it be started asynchronously,
34
+
- Should it be forwarded to another host,
35
+
- How arguments should be marshalled, and so on.
36
+
37
+
A new thread is created by [xapi/server_helpers.ml](https://github.com/xapi-project/xen-api/blob/7ac88b90e762065c5ebb94a8ea61c61bdbf62c5c/ocaml/xapi/server_helpers.ml#L55)
38
+
to handle the command asynchronously. The helper thread checks if
35
39
the command should be passed to the [message forwarding](https://github.com/xapi-project/xen-api/blob/master/ocaml/xapi/message_forwarding.ml)
36
-
layer in order to be executed on another host (the destination) or locally if
37
-
we are already at the right place.
40
+
layer in order to be executed on another host (the destination) or locally (if
41
+
it is already at the destination host).
38
42
39
43
It will finally reach [xapi/api_server.ml](https://github.com/xapi-project/xen-api/blob/7ac88b90e762065c5ebb94a8ea61c61bdbf62c5c/ocaml/xapi/api_server.ml#L242) that will take the action
40
44
of posted a command to the message broker [message switch](https://github.com/xapi-project/xen-api/tree/master/ocaml/message-switch).
@@ -43,34 +47,38 @@ XAPI daemons. In the case of the migration this message sends by **XAPI** will b
43
47
consumed by the [xenopsd](https://github.com/xapi-project/xen-api/tree/master/ocaml/xenopsd)
44
48
daemon that will do the job of migrating the VM.
45
49
46
-
#The migration of the VM
50
+
## Overview
47
51
48
52
The migration is an asynchronous task and a thread is created to handle this task.
49
-
The tasks's reference is returned to the client, which can then check
53
+
The task reference is returned to the client, which can then check
50
54
its status until completion.
51
55
52
-
As we see in the introduction the[xenopsd](https://github.com/xapi-project/xen-api/tree/master/ocaml/xenopsd)
53
-
daemon will pop the operation
56
+
As shown in the introduction,[xenopsd](https://github.com/xapi-project/xen-api/tree/master/ocaml/xenopsd)
The entities that need to be migrated are: *VDI*, *VIF*, *VGPU* and *PCI* components.
61
67
62
-
During the migration process the destination domain will be built with the same
63
-
uuid than the original VM but the last part of the UUID will be
68
+
During the migration process, the destination domain will be built with the same
69
+
UUID as the original VM, except that the last part of the UUID will be
64
70
`XXXXXXXX-XXXX-XXXX-XXXX-000000000001`. The original domain will be removed using
65
71
`XXXXXXXX-XXXX-XXXX-XXXX-000000000000`.
66
72
67
-
There are some points called *hooks* at which `xenopsd` can execute some script.
68
-
Before starting a migration a command is send to the original domain to execute
69
-
a pre migrate script if it exists.
73
+
## Preparing VM migration
70
74
71
-
Before starting the migration a command is sent to Qemu using the Qemu Machine Protocol (QMP)
75
+
At specific places, `xenopsd` can execute *hooks* to run scripts.
76
+
In case a pre-migrate script is in place, a command to run this script
77
+
is sent to the original domain.
78
+
79
+
Likewise, a command is sent to Qemu using the Qemu Machine Protocol (QMP)
72
80
to check that the domain can be suspended (see [xenopsd/xc/device_common.ml](https://github.com/xapi-project/xen-api/blob/master/ocaml/xenopsd/xc/device_common.ml)).
73
-
After checking with Qemu that the VM is suspendable we can start the migration.
81
+
After checking with Qemu that the VM is can be suspended, the migration can begin.
74
82
75
83
## Importing metadata
76
84
@@ -82,38 +90,34 @@ Once imported, it will give us a reference id and will allow building the new do
82
90
on the destination using the temporary VM uuid `XXXXXXXX-XXXX-XXXX-XXXX-000000000001`
83
91
where `XXX...` is the reference id of the original VM.
84
92
85
-
## Setting memory
93
+
## Memory setup
86
94
87
-
One of the first thing to do is to set up the memory. The backend will check that there
88
-
is no ballooning operation in progress. At this point the migration can fail if a
89
-
ballooning operation is in progress and takes too much time.
95
+
One of the first steps the setup of the VM's memory: The backend checks that there
96
+
is no ballooning operation in progress. If so, the migration could fail.
90
97
91
98
Once memory has been checked, the daemon will get the state of the VM (running, halted, ...) and
92
-
information about the VM is retrieved by the backend like the maximum memory the domain
93
-
can consume but also information about quotas for example.
94
-
The backend retrieves this information from the Xenstore.
99
+
The backend retrieves the domain's platform data (memory, vCPUs setc) from the Xenstore.
95
100
96
101
Once this is complete, we can restore VIF and create the domain.
97
102
98
103
The synchronisation of the memory is the first point of synchronisation and everything
99
104
is ready for VM migration.
100
105
101
-
## VM Migration
106
+
## Destination VM setup
102
107
103
108
After receiving memory we can set up the destination domain. If we have a vGPU we need to kick
104
-
off its migration process. We will need to wait the acknowledge that indicates that the entry
105
-
for the GPU has been well initialized. before starting the main VM migration.
109
+
off its migration process. We will need to wait for the acknowledgement that the
110
+
GPU entry has been successfully initialized before starting the main VM migration.
106
111
107
-
Their is a mechanism of handshake for synchronizing between the source and the
108
-
destination. Using the handshake protocol the receiver inform the sender of the
109
-
request that everything has been setup and ready to save/restore.
112
+
The receiver informs the sender using a handshake protocol
113
+
that everything is set up and ready for save/restore.
110
114
111
-
###VM restore
115
+
##Destination VM restore
112
116
113
117
VM restore is a low level atomic operation [VM.restore](https://github.com/xapi-project/xen-api/blob/7ac88b90e762065c5ebb94a8ea61c61bdbf62c5c/ocaml/xenopsd/xc/xenops_server_xen.ml#L2684).
114
118
This operation is represented by a function call to [backend](https://github.com/xapi-project/xen-api/blob/7ac88b90e762065c5ebb94a8ea61c61bdbf62c5c/ocaml/xenopsd/xc/domain.ml#L1540).
115
119
It uses **Xenguest**, a low-level utility from XAPI toolstack, to interact with the Xen hypervisor
116
-
and libxc for sending a request of migration to the **emu-manager**.
120
+
and `libxc` for sending a migration request to the **emu-manager**.
117
121
118
122
After sending the request results coming from **emu-manager** are collected
119
123
by the main thread. It blocks until results are received.
@@ -123,16 +127,14 @@ transitions for the devices and handling the message passing for the VM as
123
127
it's moved between hosts. This includes making sure that the state of the
124
128
VM's virtual devices, like disks or network interfaces, is correctly moved over.
125
129
126
-
### VM renaming
130
+
##Destination VM rename
127
131
128
-
Once all operations are done we can rename the VM on the target from its temporary
129
-
name to its real UUID. This operation is another lowlevel atomic one
132
+
Once all operations are done, `xenopsd` renames the target VM from its temporary
133
+
name to its real UUID. This operation is a low-level atomic
that will take care of updating the xenstore on the destination.
132
-
133
-
The next step is the restauration of devices and unpause the domain.
135
+
which takes care of updating the Xenstore on the destination host.
134
136
135
-
###Restoring remaining devices
137
+
## Restoring devices
136
138
137
139
Restoring devices starts by activating VBD using the low level atomic operation
138
140
[VBD.set_active](https://github.com/xapi-project/xen-api/blob/7ac88b90e762065c5ebb94a8ea61c61bdbf62c5c/ocaml/xenopsd/xc/xenops_server_xen.ml#L3674). It is an update of Xenstore. VBDs that are read-write must
@@ -143,39 +145,51 @@ is called. VDI are attached and activate.
143
145
Next devices are VIFs that are set as active [VIF.set_active](https://github.com/xapi-project/xen-api/blob/7ac88b90e762065c5ebb94a8ea61c61bdbf62c5c/ocaml/xenopsd/xc/xenops_server_xen.ml#L4296) and plug [VIF.plug](https://github.com/xapi-project/xen-api/blob/7ac88b90e762065c5ebb94a8ea61c61bdbf62c5c/ocaml/xenopsd/xc/xenops_server_xen.ml#L4394).
144
146
If there are VGPUs we will set them as active now using the atomic [VGPU.set_active](https://github.com/xapi-project/xen-api/blob/7ac88b90e762065c5ebb94a8ea61c61bdbf62c5c/ocaml/xenopsd/xc/xenops_server_xen.ml#L3490).
145
147
146
-
We are almost done. The next step is to create the device model
147
-
148
-
#### create device model
148
+
### Creating the device model
149
149
150
-
Create device model is done by using the atomic operation [VM.create_device_model](https://github.com/xapi-project/xen-api/blob/7ac88b90e762065c5ebb94a8ea61c61bdbf62c5c/ocaml/xenopsd/xc/xenops_server_xen.ml#L2375). This
151
-
will configure **qemu-dm** and started. This allows to manage PCI devices.
Unpause is done by managing the state of the domain using bindings to [xenctrl](https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libs/ctrl/xc_domain.c;h=f2d9d14b4d9f24553fa766c5dcb289f88d684bb0;hb=HEAD#l76).
165
-
Once hypervisor has unpaused the domain some actions can be requested using [VM.set_domain_action_request](https://github.com/xapi-project/xen-api/blob/7ac88b90e762065c5ebb94a8ea61c61bdbf62c5c/ocaml/xenopsd/xc/xenops_server_xen.ml#L3172).
166
-
It is a path in xenstore. By default no action is done but a reboot can be for example
167
-
initiated.
165
+
## Cleanup
168
166
169
-
Previously we spoke about some points called *hooks* at which `xenopsd` can execute some script. There
170
-
is also a hook to run a post migrate script. After the execution of the script if there is one
171
-
the migration is almost done. The last step is a handshake to seal the success of the migration
0 commit comments