Skip to content

Add events for dart mcp server tool invocations #2112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

jakemac53
Copy link
Contributor

This is more a proposal for potential new analytics events than a PR intended to be merged right now.

Specifically, we should consider whether we want a more general purpose event type, or specific events for different MCP server interactions.

For example we do expose runtime errors as both a tool and a resource currently, depending on the client support. Do we want a separate event to capture access to resources, or should we try and make this more general so we can add that kind of event later on without changes to this package?

Copy link

github-actions bot commented Jun 17, 2025

Package publishing

Package Version Status Publish tag (post-merge)
package:bazel_worker 1.1.3 already published at pub.dev
package:benchmark_harness 2.4.0-wip WIP (no publish necessary)
package:boolean_selector 2.1.2 already published at pub.dev
package:browser_launcher 1.1.3 already published at pub.dev
package:cli_config 0.2.1-wip WIP (no publish necessary)
package:cli_util 0.4.2 already published at pub.dev
package:clock 1.1.3-wip WIP (no publish necessary)
package:code_builder 4.10.2-wip WIP (no publish necessary)
package:coverage 1.14.1 already published at pub.dev
package:csslib 1.0.2 already published at pub.dev
package:extension_discovery 2.1.0 already published at pub.dev
package:file 7.0.2-wip WIP (no publish necessary)
package:file_testing 3.1.0-wip WIP (no publish necessary)
package:glob 2.1.3 already published at pub.dev
package:graphs 2.3.3-wip WIP (no publish necessary)
package:html 0.15.6 already published at pub.dev
package:io 1.1.0-wip WIP (no publish necessary)
package:json_rpc_2 4.0.0 already published at pub.dev
package:markdown 7.3.1-wip WIP (no publish necessary)
package:mime 2.0.0 already published at pub.dev
package:oauth2 2.0.4-wip WIP (no publish necessary)
package:package_config 2.3.0-wip WIP (no publish necessary)
package:pool 1.5.2-wip WIP (no publish necessary)
package:process 5.0.4 already published at pub.dev
package:pub_semver 2.2.0 already published at pub.dev
package:pubspec_parse 1.5.0 already published at pub.dev
package:source_map_stack_trace 2.1.3-wip WIP (no publish necessary)
package:source_maps 0.10.14-wip WIP (no publish necessary)
package:source_span 1.10.1 already published at pub.dev
package:sse 4.1.8 already published at pub.dev
package:stack_trace 1.12.1 already published at pub.dev
package:stream_channel 2.1.4 already published at pub.dev
package:stream_transform 2.1.2-wip WIP (no publish necessary)
package:string_scanner 1.4.1 already published at pub.dev
package:term_glyph 1.2.3-wip WIP (no publish necessary)
package:test_reflective_loader 0.3.0 ready to publish test_reflective_loader-v0.3.0
package:timing 1.0.2 already published at pub.dev
package:unified_analytics 8.0.2 ready to publish unified_analytics-v8.0.2
package:watcher 1.1.2 already published at pub.dev
package:yaml 3.1.3 already published at pub.dev
package:yaml_edit 2.2.2 already published at pub.dev

Documentation at https://github.com/dart-lang/ecosystem/wiki/Publishing-automation.

Copy link

github-actions bot commented Jun 17, 2025

PR Health

Breaking changes ⚠️
Package Change Current Version New Version Needed Version Looking good?
unified_analytics Breaking 8.0.1 8.0.2 9.0.0
Got "8.0.2" expected >= "9.0.0" (breaking changes)
⚠️

This check can be disabled by tagging the PR with skip-breaking-check.

Changelog Entry ✔️
Package Changed Files

Changes to files need to be accounted for in their respective changelogs.

Coverage ⚠️
File Coverage
pkgs/unified_analytics/lib/src/constants.dart 💔 Not covered
pkgs/unified_analytics/lib/src/enums.dart 💚 100 %
pkgs/unified_analytics/lib/src/event.dart 💚 98 % ⬆️ 0 %

This check for test coverage is informational (issues shown here will not fail the PR).

This check can be disabled by tagging the PR with skip-coverage-check.

API leaks ⚠️

The following packages contain symbols visible in the public API, but not exported by the library. Export these symbols or remove them from your publicly visible API.

Package Leaked API symbols
unified_analytics Condition
PersistedSurvey
GAClient
UserProperty

This check can be disabled by tagging the PR with skip-leaking-check.

License Headers ✔️
// Copyright (c) 2025, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
Files
no missing headers

All source files should start with a license header.

Unrelated files missing license headers
Files
pkgs/bazel_worker/benchmark/benchmark.dart
pkgs/bazel_worker/example/client.dart
pkgs/bazel_worker/example/worker.dart
pkgs/benchmark_harness/integration_test/perf_benchmark_test.dart
pkgs/boolean_selector/example/example.dart
pkgs/clock/lib/clock.dart
pkgs/clock/lib/src/clock.dart
pkgs/clock/lib/src/default.dart
pkgs/clock/lib/src/stopwatch.dart
pkgs/clock/lib/src/utils.dart
pkgs/clock/test/clock_test.dart
pkgs/clock/test/default_test.dart
pkgs/clock/test/stopwatch_test.dart
pkgs/clock/test/utils.dart
pkgs/coverage/lib/src/coverage_options.dart
pkgs/html/example/main.dart
pkgs/html/lib/dom.dart
pkgs/html/lib/dom_parsing.dart
pkgs/html/lib/html_escape.dart
pkgs/html/lib/parser.dart
pkgs/html/lib/src/constants.dart
pkgs/html/lib/src/encoding_parser.dart
pkgs/html/lib/src/html_input_stream.dart
pkgs/html/lib/src/list_proxy.dart
pkgs/html/lib/src/query_selector.dart
pkgs/html/lib/src/token.dart
pkgs/html/lib/src/tokenizer.dart
pkgs/html/lib/src/treebuilder.dart
pkgs/html/lib/src/utils.dart
pkgs/html/test/dom_test.dart
pkgs/html/test/parser_feature_test.dart
pkgs/html/test/parser_test.dart
pkgs/html/test/query_selector_test.dart
pkgs/html/test/selectors/level1_baseline_test.dart
pkgs/html/test/selectors/level1_lib.dart
pkgs/html/test/selectors/selectors.dart
pkgs/html/test/support.dart
pkgs/html/test/tokenizer_test.dart
pkgs/html/test/trie_test.dart
pkgs/html/tool/generate_trie.dart
pkgs/pubspec_parse/test/git_uri_test.dart
pkgs/stack_trace/example/example.dart
pkgs/watcher/test/custom_watcher_factory_test.dart
pkgs/yaml_edit/example/example.dart

@kenzieschmoll
Copy link
Contributor

Cross linking tracking issue: #2112

@kenzieschmoll
Copy link
Contributor

Specifically, we should consider whether we want a more general purpose event type, or specific events for different MCP server interactions.

For example we do expose runtime errors as both a tool and a resource currently, depending on the client support. Do we want a separate event to capture access to resources, or should we try and make this more general so we can add that kind of event later on without changes to this package?

We could add a parameter type instead where type is 'tool', 'resource', 'prompt', etc. That would make it easier to tweak analytics events down the road without needing to make a change to this package. We took a more general approach for DevTools analytics sent through the unified_analytics package and it made development much simpler to not have to change & publish this package just to add a new event.

@jakemac53
Copy link
Contributor Author

We could add a parameter type instead where type is 'tool', 'resource', 'prompt', etc. That would make it easier to tweak analytics events down the road without needing to make a change to this package. We took a more general approach for DevTools analytics sent through the unified_analytics package and it made development much simpler to not have to change & publish this package just to add a new event.

Yeah, I just don't know if we need more specific types or not, if generic events are fine I would lean towards that direction.

@kenzieschmoll
Copy link
Contributor

Generic events are fine to use. We can just pass some event label in the data and then use that to look it up in queries. Take a look at devtoolsEvent for an example. We send every DevTools analytics event through this single unified analytics event.

@jakemac53
Copy link
Contributor Author

Ok, I made the events more generic and also added a field for the MCP server version.

@jakemac53
Copy link
Contributor Author

(note that I will update tests once we agree on a structure for these events)

@jakemac53
Copy link
Contributor Author

jakemac53 commented Jun 17, 2025

Fwiw, when I tested using these events in the actual MCP server, I am getting errors because of the CustomMetrics field.

It doesn't appear that Maps are actually allowed, so this whole CustomMetrics class seems like it needs to be deprecated.

We could encode all the event data as a String... although that seems less than ideal. May want to move as many shared components out as possible (like elapsed milliseconds).

Nevermind, I see the DevTools strategy is just to flatten all the fields directly into the parent map, that works.

Copy link
Contributor

@kenzieschmoll kenzieschmoll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once this is approved from a product perspective @anderdobo

@jakemac53 jakemac53 changed the title [WIP] add events for dart mcp server tool invocations Add events for dart mcp server tool invocations Jun 23, 2025
@@ -55,6 +55,11 @@ enum DashEvent {
description: 'Pub package resolution details',
toolOwner: DashTool.dartTool,
),
dartMCPEvent(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I assume I don't have to consider this breaking since it isn't exported publicly.

@jakemac53
Copy link
Contributor Author

Ok, I just pushed up the test and pubspec/changelog updates, PTAL.

Are we ok to publish this right after the merge?

@jakemac53 jakemac53 requested a review from kenzieschmoll June 23, 2025 15:28
///
/// The [serverVersion] is the version of the Dart MCP server.
///
/// The [type] identifies the kind of event this is, and [eventData] is the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have a list of the expected event types? should we use an enum?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will live in the dart_mcp_server package

@jakemac53 jakemac53 requested a review from kenzieschmoll June 23, 2025 22:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants