Skip to content

Resurrect RRD scripts #6295

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Feb 13, 2025
Merged

Resurrect RRD scripts #6295

merged 3 commits into from
Feb 13, 2025

Conversation

last-genius
Copy link
Contributor

Resurrects two previously deleted scripts: metrics.py and metricsgraph.py.

The first one dumps a human-readable list of all host, VM, VIF, PIF, VBD-related metrics for quick troubleshooting.

metricsgraph.py creates a textual representation of the graph below, representing relationships between and basic characteristics/metrics of objects:

image

These are both installed alongside other Python libexec tools. I haven't altered what the scripts do themselves beyond bringing Python and API usage up to date, if anyone feels like these tools could be changed in some way to better serve their purpose (quick diagnostics for support/potentially part of the bugtool), please let me know.

python3: Add previously unused API classes to Python stubs used durin…
eed4377 
…g testing

Otherwise the CI will complain during pre-commit checks as it will fail to find
methods and attributes on 'None' objects.

Signed-off-by: Andrii Sultanov <[email protected]>
@lindig
Copy link
Contributor

lindig commented Feb 12, 2025

Could these go into /opt/xensource/debug/ to indicate that these are tools for troubleshooting and not enterprise-grade tools?

contificate
contificate approved these changes Feb 12, 2025
View reviewed changes
Copy link
Contributor

@contificate contificate left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extreme nitpicks (feel free to ignore entirely):

  • Format strings (f-strings) are preferable over % with format specifiers.
  • shape=rect on nodes would probably look better, visually.

Andrii Sultanov added 2 commits February 12, 2025 14:17
python3: Resurrect metrics.py helper script
6617803 
This script was previously deleted in xapi-project#3949, update it to Python 3 and the new
API methods (VIF.get_metrics and the like were removed years ago), get rid of
globals and remove the sanitycheck library usage.

Install it alongside other libexec python scripts.

The script prints out RRD metrics in the following format:
```
$ ./metrics.py
------------
 Host Metrics
 ------------
name_label  : 'lcy2-dt29'
metrics  : {'last_updated': <DateTime '20250107T14:48:01Z' at 0x7f13e3a264a8>,
 'live': True,
 'memory_free': '22100557824',
 'memory_total': '34172760064',
 'other_config': {},
 'uuid': '14cbb5db-63d8-f631-7179-6011cc578305'}
host_cpus  : [{'family': '6',
  'features': '',
  'flags': 'fpu de tsc msr pae mce cx8 apic sep mca cmov pat clflush acpi mmx '
           'fxsr sse sse2 ss ht syscall nx rdtscp lm constant_tsc rep_good '
           'nopl nonstop_tsc cpuid pni pclmulqdq monitor est ssse3 fma cx16 '
           'sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor '
           'lahf_lm abm 3dnowprefetch cpuid_fault ssbd ibrs ibpb stibp '
           'fsgsbase bmi1 avx2 bmi2 erms rdseed adx clflushopt xsaveopt xsavec '
           'xgetbv1 arch_capabilities',
  'host': 'OpaqueRef:5cdd5f91-b5e2-ba4f-09df-0f4ef9b77ef4',
  'model': '158',
  'modelname': 'Intel(R) Xeon(R) CPU E3-1230 v6 @ 3.50GHz',
  'number': '7',
  'other_config': {},
  'speed': '3503',
  'stepping': '9',
  'utilisation': 0.0,
  'uuid': '99af3b01-15c3-a40b-16bb-2f5f47bf2528',
  'vendor': 'GenuineIntel'},
...
------------
-----------------------
 Virtual Machine Metrics
 -----------------------
name_label  : 'CentOS Stream 9 (1)'
metrics  : {'VCPUs_CPU': {},
 'VCPUs_flags': {},
 'VCPUs_number': '1',
 'VCPUs_params': {},
 'VCPUs_utilisation': {'0': 0.0},
 'current_domain_type': 'hvm',
 'hvm': True,
 'install_time': <DateTime '20250108T09:01:39Z' at 0x7f13e3a34908>,
 'last_updated': <DateTime '19700101T00:00:00Z' at 0x7f13e3cc6390>,
 'memory_actual': '4294955008',
 'nested_virt': False,
 'nomigrate': False,
 'other_config': {},
 'start_time': <DateTime '20250108T09:01:47Z' at 0x7f13e3a348d0>,
 'state': [],
 'uuid': '40cb2d92-f633-382e-38a5-276cae495834'}
guest_metrics  : {'PV_drivers_detected': True,
 'PV_drivers_up_to_date': True,
 'PV_drivers_version': {},
 'can_use_hotplug_vbd': 'unspecified',
 'can_use_hotplug_vif': 'unspecified',
 'disks': {},
 'last_updated': <DateTime '20250108T09:03:00Z' at 0x7f13e3a26320>,
 'live': True,
 'memory': {},
 'netbios_name': {},
 'networks': {},
 'os_version': {},
 'other': {'feature-poweroff': '1',
           'feature-reboot': '1',
           'feature-suspend': '1',
           'feature-vcpu-hotplug': '1',
           'has-vendor-device': '0',
           'platform-feature-multiprocessor-suspend': '1',
           'platform-feature-xs_reset_watches': '1'},
 'other_config': {},
 'uuid': '6829a282-7f29-1749-d148-eeda54bbecd4'}
--
name_label  : 'Control domain on host: lcy2-dt29'
metrics  : {'VCPUs_CPU': {},
 'VCPUs_flags': {},
 'VCPUs_number': '8',
 'VCPUs_params': {},
 'VCPUs_utilisation': {'1': 0.0,
                       '2': 0.0,
                       '3': 0.0,
                       '4': 0.0,
                       '5': 0.0,
                       '6': 0.0,
                       '7': 0.0,
                       '8': 0.0},
 'current_domain_type': 'pv',
 'hvm': False,
 'install_time': <DateTime '19700101T00:00:00Z' at 0x7f13e3a34ba8>,
 'last_updated': <DateTime '19700101T00:00:00Z' at 0x7f13e3a34be0>,
 'memory_actual': '2785017856',
 'nested_virt': False,
 'nomigrate': False,
 'other_config': {},
 'start_time': <DateTime '19700101T00:00:00Z' at 0x7f13e3a34b70>,
 'state': [],
 'uuid': 'f7385e35-f114-c165-d66c-e9b01ec47ce9'}
guest_metrics  : 'NULL'
--
name_label  : 'Windows 10 (64-bit) (1)'
metrics  : {'VCPUs_CPU': {},
 'VCPUs_flags': {},
 'VCPUs_number': '2',
 'VCPUs_params': {},
 'VCPUs_utilisation': {'0': 0.0},
 'current_domain_type': 'hvm',
 'hvm': True,
 'install_time': <DateTime '20250108T09:01:24Z' at 0x7f13e3a34da0>,
 'last_updated': <DateTime '19700101T00:00:00Z' at 0x7f13e3a264e0>,
 'memory_actual': '4297043968',
 'nested_virt': False,
 'nomigrate': False,
 'other_config': {},
 'start_time': <DateTime '20250108T09:07:47Z' at 0x7f13e3a34d68>,
 'state': [],
 'uuid': '548a4d19-5db3-bce7-1a18-ab9affc2e9ee'}
guest_metrics  : {'PV_drivers_detected': True,
 'PV_drivers_up_to_date': True,
 'PV_drivers_version': {},
 'can_use_hotplug_vbd': 'yes',
 'can_use_hotplug_vif': 'yes',
 'disks': {},
 'last_updated': <DateTime '20250108T09:11:57Z' at 0x7f13e3a26400>,
 'live': True,
 'memory': {},
 'netbios_name': {},
 'networks': {'0/ip': '10.71.58.127',
              '0/ipv4/0': '10.71.58.127',
              '0/ipv6/0': 'fe80:0000:0000:0000:70ad:88ff:febb:643d'},
 'os_version': {},
 'other': {'data-cant-suspend-reason': '{"error":{"class":"GenericError","desc":"State '
                                       'blocked by non-migratable device '
                                       '\'0000:00:07.0/nvme\'","data":{}},"id":"qmp-000013-4"}',
           'feature-balloon': '1',
           'feature-laptop-slate-mode': '1',
           'feature-poweroff': '1',
           'feature-reboot': '1',
           'feature-s3': '1',
           'feature-s4': '1',
           'feature-shutdown': '1',
           'has-vendor-device': '1',
           'platform-feature-multiprocessor-suspend': '1',
           'platform-feature-xs_reset_watches': '1'},
 'other_config': {},
 'uuid': 'dcd51a5f-3017-a7ec-3d35-7648366b887e'}
-----------------------
-----------
 VIF metrics
 -----------
name_label  : 'VIF connecting "NPRI bond of 0 1" to "CentOS Stream 9 (1)"'
metrics  : {'vif_0_rx': 0.0,
 'vif_0_rx_errors': 0.0,
 'vif_0_tx': 0.0,
 'vif_0_tx_errors': 0.0}
--
name_label  : 'VIF connecting "NPRI bond of 0 1" to "Windows 10 (64-bit) (1)"'
metrics  : {'vif_0_rx': 0.0,
 'vif_0_rx_errors': 0.0,
 'vif_0_tx': 0.0,
 'vif_0_tx_errors': 0.0}
-----------
-----------
 VBD Metrics
 -----------
name_label  : ('VBD connecting "CentOS Stream 9 (1)" to '
 '"CentOS-Stream-8-x86_64-latest-boot.iso"')
metrics  : {'vbd_xvdd_avgqu_sz': 0.0,
 'vbd_xvdd_inflight': 0.0,
 'vbd_xvdd_io_throughput_read': 0.0,
 'vbd_xvdd_io_throughput_total': 0.0,
 'vbd_xvdd_io_throughput_write': 0.0,
 'vbd_xvdd_iops_read': 0.0,
 'vbd_xvdd_iops_total': 0.0,
 'vbd_xvdd_iops_write': 0.0,
 'vbd_xvdd_iowait': 0.0,
 'vbd_xvdd_latency': 0.0,
 'vbd_xvdd_read': 0.0,
 'vbd_xvdd_read_latency': 0.0,
 'vbd_xvdd_write': 0.0,
 'vbd_xvdd_write_latency': 0.0}
--
name_label  : 'VBD connecting "Windows 10 (64-bit) (1)" to "win10-x64_uefi.iso"'
metrics  : {'vbd_xvdd_avgqu_sz': 0.0,
 'vbd_xvdd_inflight': 0.0,
 'vbd_xvdd_io_throughput_read': 0.0,
 'vbd_xvdd_io_throughput_total': 0.0,
 'vbd_xvdd_io_throughput_write': 0.0,
 'vbd_xvdd_iops_read': 0.0,
 'vbd_xvdd_iops_total': 0.0,
 'vbd_xvdd_iops_write': 0.0,
 'vbd_xvdd_iowait': 0.0,
 'vbd_xvdd_latency': 0.0,
 'vbd_xvdd_read': 0.0,
 'vbd_xvdd_read_latency': 0.0,
 'vbd_xvdd_write': 0.0,
 'vbd_xvdd_write_latency': 0.0}
--
name_label  : 'VBD connecting "Windows 10 (64-bit) (1)" to "Windows 10 (64-bit) (1) 0"'
metrics  : {'vbd_xvda_avgqu_sz': 0.0,
 'vbd_xvda_inflight': 0.0,
 'vbd_xvda_io_throughput_read': 0.0,
 'vbd_xvda_io_throughput_total': 0.0,
 'vbd_xvda_io_throughput_write': 0.0,
 'vbd_xvda_iops_read': 0.0,
 'vbd_xvda_iops_total': 0.0,
 'vbd_xvda_iops_write': 0.0,
 'vbd_xvda_iowait': 0.0,
 'vbd_xvda_latency': 0.0,
 'vbd_xvda_read': 0.0,
 'vbd_xvda_read_latency': 0.0,
 'vbd_xvda_write': 0.0,
 'vbd_xvda_write_latency': 0.0}
--
name_label  : 'VBD connecting "CentOS Stream 9 (1)" to "CentOS Stream 9 (1) 0"'
metrics  : {'vbd_xvda_avgqu_sz': 0.0,
 'vbd_xvda_inflight': 0.0,
 'vbd_xvda_io_throughput_read': 0.0,
 'vbd_xvda_io_throughput_total': 0.0,
 'vbd_xvda_io_throughput_write': 0.0,
 'vbd_xvda_iops_read': 0.0,
 'vbd_xvda_iops_total': 0.0,
 'vbd_xvda_iops_write': 0.0,
 'vbd_xvda_iowait': 0.0,
 'vbd_xvda_latency': 0.0,
 'vbd_xvda_read': 0.0,
 'vbd_xvda_read_latency': 0.0,
 'vbd_xvda_write': 0.0,
 'vbd_xvda_write_latency': 0.0}
-----------
---------------
 Network Metrics
 ---------------
name_label  : 'NPRI bond of 0 1'
---------------
-----------
 PIF Metrics
 -----------
name_label  : 'bond0 on lcy2-dt29'
metrics  : {'pif_bond0_rx': 0.0,
 'pif_bond0_rx_errors': 0.0,
 'pif_bond0_tx': 0.0,
 'pif_bond0_tx_errors': 0.0}
-----------
Active Objects
host_metrics ['lcy2-dt29']
vm_metrics ['CentOS Stream 9 (1)', 'Control domain on host: lcy2-dt29', 'Windows 10 (64-bit) (1)']
vif_metrics ['VIF connecting "NPRI bond of 0 1" to "CentOS Stream 9 (1)"', 'VIF connecting "NPRI bond of 0 1" to "Windows 10 (64-bit) (1)"']
vbd_metrics ['VBD connecting "CentOS Stream 9 (1)" to "CentOS-Stream-8-x86_64-latest-boot.iso"', 'VBD connecting "Windows 10 (64-bit) (1)" to "win10-x64_uefi.iso"', 'VBD connecting "Windows 10 (64-bit) (1)" to "Windows 10 (64-bit) (1) 0"', 'VBD connecting "CentOS Stream 9 (1)" to "CentOS Stream 9 (1) 0"']
network_metrics ['NPRI bond of 0 1']
pif_metrics ['bond0 on lcy2-dt29']
```

Signed-off-by: Andrii Sultanov <[email protected]>
python3: Resurrect a metricsgraph.py helper script
7269b09 
This script was previously deleted in xapi-project#3949, update it to Python 3 and the new
API methods (VIF.get_metrics and the like were removed years ago), get rid of
globals and remove the sanitycheck library usage.

Install it alongside other libexec python scripts.

The script outputs a short list of active objects and their graph in the graphviz
format (it could be turned into an image with the help of various graphviz
utilities like `echo digraph active_objects {...} | dot -Tsvg > output.svg`)

```
$ ./metricsgraph.py
/*
hosts : lcy2-dt29
*/
/*
resident VMs : CentOS Stream 9 (1), Control domain on host: lcy2-dt29, Windows 10 (64-bit) (1)
*/
/*
active VIFs : OpaqueRef:051f732b-4897-17ab-1a03-9ab2e90a6d8e, OpaqueRef:b7291919-2f60-83a0-7777-96ffd6a73e32
*/
/*
active VBDs : OpaqueRef:21f94d40-1be3-1dee-d394-7c18eaaad8b4, OpaqueRef:44984583-9938-1177-e7e5-dbc4bd23f51e, OpaqueRef:6de310eb-2b0b-85c7-cd5b-d3ccbe46f3d8, OpaqueRef:543537d3-ec19-b67d-7422-545894cf6727
*/
/*
active networks : NPRI bond of 0 1
*/
/*
active PIFs : OpaqueRef:4d293e68-1c2a-c6a8-bdd9-823abca394ba
*/
digraph active_objects {
node [shape="rect"];
"OpaqueRef:5cdd5f91-b5e2-ba4f-09df-0f4ef9b77ef4" [label="lcy2-dt29\ncpus=0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00\nmemory=0.65"];
"OpaqueRef:1d340059-3a7b-bac6-9d90-26e87ecc4a2a" [label="CentOS Stream 9 (1)\ncpus=0.00\nmemory=4095.98828125M"];
"OpaqueRef:f76036a2-17d8-c6fa-7959-1e5b10c50d3b" [label="Control domain on host: lcy2-dt29\ncpus=0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00\nmemory=2656.0M"];
"OpaqueRef:c613d702-b8d7-2313-34b5-2683accae3ed" [label="Windows 10 (64-bit) (1)\ncpus=0.00\nmemory=4097.98046875M"];
"OpaqueRef:051f732b-4897-17ab-1a03-9ab2e90a6d8e" [label="vif\nread=0.00k\nwrite=0.00k"];
"OpaqueRef:b7291919-2f60-83a0-7777-96ffd6a73e32" [label="vif\nread=0.00k\nwrite=0.00k"];
"OpaqueRef:21f94d40-1be3-1dee-d394-7c18eaaad8b4" [label="vbd\nread=0.00k\nwrite=0.00k"];
"OpaqueRef:44984583-9938-1177-e7e5-dbc4bd23f51e" [label="vbd\nread=0.00k\nwrite=0.00k"];
"OpaqueRef:6de310eb-2b0b-85c7-cd5b-d3ccbe46f3d8" [label="vbd\nread=0.00k\nwrite=0.00k"];
"OpaqueRef:543537d3-ec19-b67d-7422-545894cf6727" [label="vbd\nread=0.00k\nwrite=0.00k"];
"OpaqueRef:4d293e68-1c2a-c6a8-bdd9-823abca394ba" [label="pif\nread=0.00k\nwrite=0.00k"];
"OpaqueRef:cf42e13c-aa9d-c604-710c-bead3587948a" [label="NPRI bond of 0 1"];
"OpaqueRef:5cdd5f91-b5e2-ba4f-09df-0f4ef9b77ef4" -> "OpaqueRef:1d340059-3a7b-bac6-9d90-26e87ecc4a2a";
"OpaqueRef:5cdd5f91-b5e2-ba4f-09df-0f4ef9b77ef4" -> "OpaqueRef:c613d702-b8d7-2313-34b5-2683accae3ed";
"OpaqueRef:5cdd5f91-b5e2-ba4f-09df-0f4ef9b77ef4" -> "OpaqueRef:f76036a2-17d8-c6fa-7959-1e5b10c50d3b";
"OpaqueRef:1d340059-3a7b-bac6-9d90-26e87ecc4a2a" -> "OpaqueRef:b7291919-2f60-83a0-7777-96ffd6a73e32";
"OpaqueRef:c613d702-b8d7-2313-34b5-2683accae3ed" -> "OpaqueRef:051f732b-4897-17ab-1a03-9ab2e90a6d8e";
"OpaqueRef:1d340059-3a7b-bac6-9d90-26e87ecc4a2a" -> "OpaqueRef:21f94d40-1be3-1dee-d394-7c18eaaad8b4";
"OpaqueRef:1d340059-3a7b-bac6-9d90-26e87ecc4a2a" -> "OpaqueRef:6de310eb-2b0b-85c7-cd5b-d3ccbe46f3d8";
"OpaqueRef:c613d702-b8d7-2313-34b5-2683accae3ed" -> "OpaqueRef:44984583-9938-1177-e7e5-dbc4bd23f51e";
"OpaqueRef:c613d702-b8d7-2313-34b5-2683accae3ed" -> "OpaqueRef:543537d3-ec19-b67d-7422-545894cf6727";
"OpaqueRef:051f732b-4897-17ab-1a03-9ab2e90a6d8e" -> "OpaqueRef:cf42e13c-aa9d-c604-710c-bead3587948a";
"OpaqueRef:b7291919-2f60-83a0-7777-96ffd6a73e32" -> "OpaqueRef:cf42e13c-aa9d-c604-710c-bead3587948a";
"OpaqueRef:cf42e13c-aa9d-c604-710c-bead3587948a" -> "OpaqueRef:4d293e68-1c2a-c6a8-bdd9-823abca394ba";
}
```

Signed-off-by: Andrii Sultanov <[email protected]>
@last-genius last-genius force-pushed the private/asultanov/rrd-scripts branch from 5d57328 to 7269b09 Compare February 12, 2025 14:18
@last-genius
Copy link
Contributor Author

Put these into /opt/xensource/debug instead, and added shape=rect too.

@lindig
Copy link
Contributor

lindig commented Feb 12, 2025

Does this need xapi.spec changes?

@last-genius
Copy link
Contributor Author

last-genius commented Feb 12, 2025 •
edited
Loading

Does this need xapi.spec changes?

Yes, I'll open a PR there as well

psafont
psafont approved these changes Feb 13, 2025
View reviewed changes
Copy link
Member

@psafont psafont left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low risk to merge since this are for debugging

@last-genius last-genius added this pull request to the merge queue Feb 13, 2025
Merged via the queue into xapi-project:master with commit e02427b Feb 13, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@psafont psafont psafont approved these changes

@contificate contificate contificate approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@last-genius @lindig @psafont @contificate