gpt4all/gpt4all-bindings/csharp/README.md

C# GPT4All

This package contains a set of C# bindings around the llmodel C-API.

Documentation

TBD

Installation

Windows and Linux builds are available on NuGet: https://www.nuget.org/packages/Gpt4All

macOS is WIP due to code signing issues, contributions are welcome.

Project Structure

gpt4all-bindings/
└── csharp                
    ├── Gpt4All               // .NET Bindigs
    ├── Gpt4All.Samples       // Sample project
    ├── build_win-msvc.ps1    // Native build scripts
    ├── build_win-mingw.ps1   
    ├── build_linux.sh        
    └── runtimes              // [POST-BUILD] Platform-specific native libraries
          ├── win-x64
          ├── ...
          └── linux-x64

Prerequisites

On Windows and Linux, building GPT4All requires the complete Vulkan SDK. You may download it from here: https://vulkan.lunarg.com/sdk/home

macOS users do not need Vulkan, as GPT4All will use Metal instead.

Local Build Instructions

Note

Tested On:

• Windows 11 22H + VS2022 (CE) x64
• Linux Ubuntu x64
• Linux Ubuntu (WSL2) x64

1. Setup the repository
2. Build the native libraries for the platform of choice (see below)
3. Build the C# Bindings (NET6+ SDK is required)

git clone --recurse-submodules https://github.com/nomic-ai/gpt4all
cd gpt4all/gpt4all-bindings/csharp

Linux

1. Setup build environment and install NET6+ SDK with the appropriate procedure for your distribution

sudo apt-get update
sudo apt-get install -y cmake build-essential
chmod +x ./build_linux.sh

2. ./build_linux.sh
3. The native libraries should be present at .\native\linux-x64

Windows - MinGW64

Additional requirements

• MinGW64
• CMAKE

1. Setup

choco install mingw
$env:Path += ";C:\ProgramData\mingw64\mingw64\bin"
choco install -y cmake --installargs 'ADD_CMAKE_TO_PATH=System'

2. Run the ./build_win-mingw.ps1 build script
3. The native libraries should be present at .\native\win-x64

Windows - MSVC

Additional requirements

• Visual Studio 2022

1. Open a terminal using the x64 Native Tools Command Prompt for VS 2022 (vcvars64.bat)
2. Run the ./build_win-msvc.ps1 build script
3. libllmodel.dll and libllama.dll should be present at .\native\win-x64

Warning

If the build fails with: 'error C7555: use of designated initializers requires at least '/std:c++20''

Modify cd gpt4all/gpt4all-backends/CMakeLists.txt adding CXX_STANDARD_20 to llmodel properties.

set_target_properties(llmodel PROPERTIES
                             VERSION ${PROJECT_VERSION}
                             CXX_STANDARD 20 # <---- ADD THIS -----------------------
                             SOVERSION ${PROJECT_VERSION_MAJOR})

C# Bindings Build Instructions

Build the Gpt4All (or Gpt4All.Samples) projects from within VisualStudio.

Try the bindings

using Gpt4All;

// load the model
var modelFactory = new ModelFactory();

using var model = modelFactory.LoadModel("./path/to/ggml-gpt4all-j-v1.3-groovy.bin");

var input = "Name 3 Colors";

// request a prediction
var result = await model.GetStreamingPredictionAsync(
    input, 
    PredictRequestOptions.Defaults);

// asynchronously print the tokens as soon as they are produces by the model
await foreach(var token in result.GetPredictionStreamingAsync())
{
    Console.Write(token);
}

Output:

gptj_model_load: loading model from 'ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
gptj_model_load: n_vocab = 50400
[...TRUNCATED...]
gptj_model_load: ggml ctx size = 5401.45 MB
gptj_model_load: kv self size  =  896.00 MB
gptj_model_load: ................................... done
gptj_model_load: model size =  3609.38 MB / num tensors = 285

Black, Blue and White