Skip to content

Python: .Net: MEVD Preview 2 APR 2025 #11028

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 87 commits into from
Apr 29, 2025
Merged
Show file tree
Hide file tree
Changes from 86 commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
bd9b83a
.Net: Remove DeleteRecordOptions and UpsertRecordOptions from MEVD (#…
roji Jan 24, 2025
2434faf
.Net MEVD: LINQ-based filtering (#10273)
roji Feb 11, 2025
a9abe9a
.Net: Refactor vector store integration tests (#10556)
roji Feb 18, 2025
3a36d40
.Net: Obsolete MEVD connectors collection factories (#10590)
roji Feb 19, 2025
10cde20
.Net: Rename VectorSearchOptions.NewFilter to Filter (#10790)
roji Mar 4, 2025
865d67e
Python: Preb1 merge from main 1 (#10805)
westey-m Mar 5, 2025
fd0c55c
Revert "Python: Preb1 merge from main 1 (#10805)"
westey-m Mar 5, 2025
be79369
Merge branch 'main' into feature-vector-data-preb1
westey-m Mar 5, 2025
0e8adb7
.Net: Hybrid Search (#10814)
westey-m Mar 6, 2025
7aa7d29
.Net: IVectorStore implementation for Azure SQL (#10623)
adamsitnik Mar 7, 2025
864b707
Enable integration tests on Windows
dmytrostruk Mar 7, 2025
9e068b2
Merge branch 'feature-vector-data-preb1' of https://github.com/micros…
dmytrostruk Mar 7, 2025
fd53ae8
Revert "Enable integration tests on Windows"
dmytrostruk Mar 7, 2025
a14ad8c
.Net MEVD: Strongly-typed property selectors (#10850)
adamsitnik Mar 7, 2025
261fec4
Updated integration test pipeline to use net8.0 target framework
dmytrostruk Mar 7, 2025
3fca31f
.Net: [Feature Branch] Vector Store Logging (#10865)
dmytrostruk Mar 12, 2025
1511ab9
Merge branch 'main' into feature-vector-data-telemetry
dmytrostruk Mar 12, 2025
ce572c8
.Net: Change Sqlite connector to accept connection string instead of …
roji Mar 14, 2025
89aab09
.Net: [TINY] Document the default thread-safety expectation for the M…
roji Mar 17, 2025
458ad3a
Merge branch 'main' into feature-vector-data-preb2
westey-m Mar 19, 2025
e896415
.Net: Obsolete the IMemoryStore where an IVectorStore implementation …
roji Mar 19, 2025
9732543
.Net: Target net462 in all MEVD connectors (#11077)
roji Mar 20, 2025
01400f8
Merge branch 'feature-vector-data-telemetry' into feature-vector-data…
dmytrostruk Mar 20, 2025
d9d601b
.Net: Merge telemetry branch to preb2 (#11095)
dmytrostruk Mar 20, 2025
49ad47c
Merge branch 'feature-vector-data-preb2' of https://github.com/micros…
dmytrostruk Mar 20, 2025
9560b51
.Net MEVD: Create indexes for relational DBs (#11233)
adamsitnik Mar 28, 2025
d4742a3
.Net: Fix InMemory collection deletion. (#11256)
dluc Mar 28, 2025
0607deb
Merge branch 'main' into feature-vector-data-preb2
westey-m Mar 28, 2025
15a2931
.Net: Preb2 fix obsolete warnings (#11268)
westey-m Mar 28, 2025
5ce4285
.Net: Start decoupling MEVD providers from SK (#11076)
roji Mar 30, 2025
77c83c5
.Net MEVD: More conformance tests coverage (#11257)
adamsitnik Mar 31, 2025
bdc1caa
Merge branch 'main' into feature-vector-data-preb2
westey-m Mar 31, 2025
a345c8b
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 1, 2025
bfee749
.Net: [MEVD] Temporarily remove logging (#11316)
dmytrostruk Apr 1, 2025
42ac3ec
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 2, 2025
ea7e0e1
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 2, 2025
da00e8d
.Net: MEVD: Remove batch from batch CRUD operation names. (#11287)
westey-m Apr 3, 2025
cb1d664
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 3, 2025
50c7ee8
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 3, 2025
a312b59
.Net: Introduce new record model (#11264)
roji Apr 3, 2025
0a1b83c
.Net: Obsolete custom mappers (#11366)
roji Apr 4, 2025
8c4518a
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 4, 2025
f703f69
.Net MEVD: Make Top mandatory argument (#11376)
adamsitnik Apr 4, 2025
802a14d
.Net: Add skip validation for hybrid search options (#11380)
westey-m Apr 4, 2025
f77e266
.Net: [MEVD] Added GetService and Metadata for vector store abstracti…
dmytrostruk Apr 7, 2025
b19fe4c
.Net: Add integration tests for no-vector models and fix bugs (#11383)
westey-m Apr 7, 2025
670ce37
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 8, 2025
1efa055
.Net: [MEVD] Updated VectorStoreException. Made implementations seale…
dmytrostruk Apr 8, 2025
89de29c
.Net: [MEVD] Improved Cosmos NoSQL initialization logic (#11446)
dmytrostruk Apr 8, 2025
747531c
.Net MEVD: Filter-only search API (without vector similarity) (#11112)
adamsitnik Apr 9, 2025
8a132c0
.Net: Rename IsFilterable to IsIndexed and IsFullTextSearchable to Is…
westey-m Apr 9, 2025
2f24a46
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 9, 2025
137e284
.Net: enforce TRecord : notnull (#11467)
adamsitnik Apr 9, 2025
3f30fae
.Net: Replace VectorStoreGenericDataModel with Dictionary<string, obj…
roji Apr 10, 2025
4fc76f2
.Net MEVD: Enable & fix integration tests on MongoDB (#11405)
damieng Apr 10, 2025
ae0fa62
.Net: Change UpsertAsync return type to IReadOnlyList<TKey> (#11402)
roji Apr 10, 2025
fe185c1
.Net: Refactor MEVD Vector Attribute and make Dimensions mandatory (#…
westey-m Apr 11, 2025
4f80c12
.Net MEVD Replace VectorSearchResults with IAsyncEnumerable (#11486)
adamsitnik Apr 12, 2025
d1a8b51
.Net: Remove custom mappers (#11530)
roji Apr 14, 2025
5867345
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 14, 2025
2338348
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 15, 2025
32f33ac
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 15, 2025
1505ad8
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 15, 2025
71da485
.Net: Merge fixes after updating to latest version of qdrant. (#11572)
westey-m Apr 16, 2025
bea493b
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 16, 2025
59be52f
.Net MEVD: fix two bugs (#11585)
adamsitnik Apr 16, 2025
4d7c045
.Net: [MEVD] Added GetRequiredService and renamed CollectionName (#11…
dmytrostruk Apr 16, 2025
473a736
.Net: [MEVD] Added support for unnamed vectors in Weaviate connector …
dmytrostruk Apr 16, 2025
4d75e8a
.Net: Add delete and exists methods for IVectorStore (#11596)
westey-m Apr 17, 2025
cd0f985
.Net MEVD Sqlite: Escaping (#11534)
adamsitnik Apr 18, 2025
4a92af9
.Net: Add tests for CRUD with no data fields and bug/test fixes (#11641)
westey-m Apr 22, 2025
3f7d5b4
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 22, 2025
b811688
Fix merge issue.
westey-m Apr 22, 2025
17ce3f0
Fix merge issues.
westey-m Apr 22, 2025
1027cc6
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 22, 2025
8b8ccd6
.Net: Introduce a LINQ filter preprocessor in MEVD (#11702)
roji Apr 26, 2025
aada9a2
.Net: Fix filtering GetAsync for Cosmos NoSQL (#11754)
roji Apr 27, 2025
39254ce
.Net: Integrate IEmbeddingGenerator (#11682)
roji Apr 28, 2025
e9ab1ca
.Net: Cleanup of SKEXP0020 flag, removing MemoryStore samples and sto…
westey-m Apr 28, 2025
2ec7d24
.Net: Remove custom mapper samples (#11742)
westey-m Apr 28, 2025
c8b129d
.Net: Tiny fixes to make MEVD integration tests pass (#11776)
roji Apr 28, 2025
a881464
.Net: Fix Integration tests and a few small bugs. (#11778)
westey-m Apr 28, 2025
87dd8b1
.Net: Small InMemory bug fix. (#11799)
westey-m Apr 29, 2025
490d644
Merge branch 'main' into feature-vector-data-preb2
westey-m Apr 29, 2025
fe58faa
.Net: Update vector store readme's (#11801)
westey-m Apr 29, 2025
454f368
.Net: Try to reduce mongodb test flakiness (#11802)
westey-m Apr 29, 2025
81f8151
Delete 00NN-hybrid-search.md
westey-m Apr 29, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
395 changes: 395 additions & 0 deletions docs/decisions/00NN-hybrid-search.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions dotnet/Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@
<PackageVersion Include="System.Net.Http" Version="4.3.4" />
<PackageVersion Include="System.Numerics.Tensors" Version="9.0.0" />
<PackageVersion Include="System.Text.Json" Version="8.0.5" />
<PackageVersion Include="System.ValueTuple" Version="4.6.1" />
<PackageVersion Include="System.Threading.Tasks.Extensions" Version="4.6.3" />
<!-- Tokenizers -->
<PackageVersion Include="Microsoft.ML.Tokenizers" Version="1.0.1" />
Expand Down
12 changes: 12 additions & 0 deletions dotnet/SK-dotnet.sln
Original file line number Diff line number Diff line change
Expand Up @@ -107,11 +107,14 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "src", "src", "{958AD708-F04
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Diagnostics", "Diagnostics", "{29E7D971-1308-4171-9872-E8E4669A1134}"
ProjectSection(SolutionItems) = preProject
src\InternalUtilities\src\Diagnostics\ActivityExtensions.cs = src\InternalUtilities\src\Diagnostics\ActivityExtensions.cs
src\InternalUtilities\src\Diagnostics\CompilerServicesAttributes.cs = src\InternalUtilities\src\Diagnostics\CompilerServicesAttributes.cs
src\InternalUtilities\src\Diagnostics\DynamicallyAccessedMembersAttribute.cs = src\InternalUtilities\src\Diagnostics\DynamicallyAccessedMembersAttribute.cs
src\InternalUtilities\src\Diagnostics\ExceptionExtensions.cs = src\InternalUtilities\src\Diagnostics\ExceptionExtensions.cs
src\InternalUtilities\src\Diagnostics\ExperimentalAttribute.cs = src\InternalUtilities\src\Diagnostics\ExperimentalAttribute.cs
src\InternalUtilities\src\Diagnostics\IsExternalInit.cs = src\InternalUtilities\src\Diagnostics\IsExternalInit.cs
src\InternalUtilities\src\Diagnostics\KernelVerify.cs = src\InternalUtilities\src\Diagnostics\KernelVerify.cs
src\InternalUtilities\src\Diagnostics\LoggingExtensions.cs = src\InternalUtilities\src\Diagnostics\LoggingExtensions.cs
src\InternalUtilities\src\Diagnostics\NullableAttributes.cs = src\InternalUtilities\src\Diagnostics\NullableAttributes.cs
src\InternalUtilities\src\Diagnostics\RequiresDynamicCodeAttribute.cs = src\InternalUtilities\src\Diagnostics\RequiresDynamicCodeAttribute.cs
src\InternalUtilities\src\Diagnostics\RequiresUnreferencedCodeAttribute.cs = src\InternalUtilities\src\Diagnostics\RequiresUnreferencedCodeAttribute.cs
Expand Down Expand Up @@ -547,6 +550,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Runtime.InProcess", "src\Ag
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Runtime.InProcess.Tests", "src\Agents\Runtime\InProcess.Tests\Runtime.InProcess.Tests.csproj", "{DA6B4ED4-ED0B-D25C-889C-9F940E714891}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "VectorData.UnitTests", "src\Connectors\VectorData.UnitTests\VectorData.UnitTests.csproj", "{AAC7B5E8-CC4E-49D0-AF6A-2B4F7B43BD84}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Expand Down Expand Up @@ -1508,6 +1513,12 @@ Global
{DA6B4ED4-ED0B-D25C-889C-9F940E714891}.Publish|Any CPU.Build.0 = Release|Any CPU
{DA6B4ED4-ED0B-D25C-889C-9F940E714891}.Release|Any CPU.ActiveCfg = Release|Any CPU
{DA6B4ED4-ED0B-D25C-889C-9F940E714891}.Release|Any CPU.Build.0 = Release|Any CPU
{AAC7B5E8-CC4E-49D0-AF6A-2B4F7B43BD84}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{AAC7B5E8-CC4E-49D0-AF6A-2B4F7B43BD84}.Debug|Any CPU.Build.0 = Debug|Any CPU
{AAC7B5E8-CC4E-49D0-AF6A-2B4F7B43BD84}.Publish|Any CPU.ActiveCfg = Debug|Any CPU
{AAC7B5E8-CC4E-49D0-AF6A-2B4F7B43BD84}.Publish|Any CPU.Build.0 = Debug|Any CPU
{AAC7B5E8-CC4E-49D0-AF6A-2B4F7B43BD84}.Release|Any CPU.ActiveCfg = Release|Any CPU
{AAC7B5E8-CC4E-49D0-AF6A-2B4F7B43BD84}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down Expand Up @@ -1713,6 +1724,7 @@ Global
{A4F05541-7D23-A5A9-033D-382F1E13D0FE} = {A70ED5A7-F8E1-4A57-9455-3C05989542DA}
{CCC909E4-5269-A31E-0BFD-4863B4B29BBB} = {A70ED5A7-F8E1-4A57-9455-3C05989542DA}
{DA6B4ED4-ED0B-D25C-889C-9F940E714891} = {A70ED5A7-F8E1-4A57-9455-3C05989542DA}
{AAC7B5E8-CC4E-49D0-AF6A-2B4F7B43BD84} = {5A7028A7-4DDF-4E4F-84A9-37CE8F8D7E89}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {FBDC56A3-86AD-4323-AA0F-201E59123B83}
Expand Down
2 changes: 2 additions & 0 deletions dotnet/docs/EXPERIMENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ You can use the following diagnostic IDs to ignore warnings or errors for a part
| SKEXP0100 | Advanced Semantic Kernel features |
| SKEXP0110 | Semantic Kernel Agents |
| SKEXP0120 | Native-AOT |
| MEVD9000 | Microsoft.Extensions.VectorData experimental user-facing APIs |
| MEVD9001 | Microsoft.Extensions.VectorData experimental connector-facing APIs |

## Experimental Features Tracking

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -199,8 +199,8 @@ public async Task OnPromptRenderAsync(PromptRenderContext context, Func<PromptRe
await collection.CreateCollectionIfNotExistsAsync();

// Search for similar prompts in cache.
var searchResults = await collection.VectorizedSearchAsync(promptEmbedding, new() { Top = 1 }, context.CancellationToken);
var searchResult = (await searchResults.Results.FirstOrDefaultAsync())?.Record;
var searchResult = (await collection.SearchEmbeddingAsync(promptEmbedding, top: 1, cancellationToken: context.CancellationToken)
.FirstOrDefaultAsync())?.Record;

// If result exists, return it.
if (searchResult is not null)
Expand Down
2 changes: 1 addition & 1 deletion dotnet/samples/Concepts/Concepts.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
<IsPackable>false</IsPackable>
<IsTestProject>true</IsTestProject>
<!-- Suppress: "Declare types in namespaces", "Require ConfigureAwait", "Experimental" -->
<NoWarn>$(NoWarn);CS8618,IDE0009,IDE1006,CA1051,CA1050,CA1707,CA1054,CA2007,VSTHRD111,CS1591,RCS1110,RCS1243,CA5394,SKEXP0001,SKEXP0010,SKEXP0020,SKEXP0040,SKEXP0050,SKEXP0060,SKEXP0070,SKEXP0101,SKEXP0110,OPENAI001,CA1724</NoWarn>
<NoWarn>$(NoWarn);CS8618,IDE0009,IDE1006,CA1051,CA1050,CA1707,CA1054,CA2007,VSTHRD111,CS1591,RCS1110,RCS1243,CA5394,SKEXP0001,SKEXP0010,SKEXP0040,SKEXP0050,SKEXP0060,SKEXP0070,SKEXP0101,SKEXP0110,OPENAI001,CA1724,MEVD9000</NoWarn>
<OutputType>Library</OutputType>
<UserSecretsId>5ee045b0-aea3-4f08-8d31-32d1a6f8fed0</UserSecretsId>
</PropertyGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ namespace Memory;
/// For example, the <a href="https://huggingface.co/cointegrated/LaBSE-en-ru">cointegrated/LaBSE-en-ru</a> model returns results as a 1 * 1 * 4 * 768 matrix, which is different from Hugging Face embedding generation service implementation.
/// To address this, a custom <see cref="HttpClientHandler"/> can be used to modify the response before sending it back.
/// </summary>
[Obsolete("The IMemoryStore abstraction is being obsoleted")]
public class HuggingFace_TextEmbeddingCustomHttpHandler(ITestOutputHelper output) : BaseTest(output)
{
public async Task RunInferenceApiEmbeddingCustomHttpHandlerAsync()
Expand Down
239 changes: 0 additions & 239 deletions dotnet/samples/Concepts/Memory/MemoryStore_CustomReadOnly.cs

This file was deleted.

Loading
Loading