You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+12-1Lines changed: 12 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -2,7 +2,7 @@
2
2
3
3

4
4
5
-
The C#/.NET binding of llama.cpp. It provides APIs to inference the LLaMa Models and deploy it on native environment or Web. It works on
5
+
The C#/.NET binding of [llama.cpp](https://github.com/ggerganov/llama.cpp). It provides APIs to inference the LLaMa Models and deploy it on native environment or Web. It works on
6
6
both Windows and Linux and does NOT require compiling the library yourself.
7
7
8
8
- Load and inference LLaMa models
@@ -20,6 +20,17 @@ Just search `LLamaSharp` in nuget package manager and install it!
20
20
PM> Install-Package LLamaSharp
21
21
```
22
22
23
+
## Simple Benchmark
24
+
25
+
Currently it's only a simple benchmark to indicate that the performance of `LLamaSharp` is close to `llama.cpp`. Experiments run on a computer
26
+
with Intel i7-12700, 3060Ti with 7B model. Note that the benchmark uses `LLamaModel` instead of `LLamaModelV1`.
27
+
28
+
#### Windows
29
+
30
+
- llama.cpp: 2.98 words / second
31
+
32
+
- LLamaSharp: 2.94 words / second
33
+
23
34
## Usages
24
35
25
36
Currently, `LLamaSharp` provides two kinds of model, `LLamaModelV1` and `LLamaModel`. Both of them works but `LLamaModel` is more recommended
0 commit comments