gpt4all/gpt4all-backend/gptj_impl.h

#ifndef GPTJ_H_I_KNOW_WHAT_I_AM_DOING_WHEN_INCLUDING_THIS_FILE
#error This file is NOT meant to be included outside of gptj.cpp. Doing so is DANGEROUS. Be sure to know what you are doing before proceeding to #define GPTJ_H_I_KNOW_WHAT_I_AM_DOING_WHEN_INCLUDING_THIS_FILE
#endif
#ifndef GPTJ_H
#define GPTJ_H

#include <string>
#include <functional>
#include <vector>
#include "llmodel.h"

struct GPTJPrivate;
class GPTJ : public LLModel {
public:
    GPTJ();
    ~GPTJ();

    bool loadModel(const std::string &modelPath) override;
    bool isModelLoaded() const override;
    size_t requiredMem(const std::string &modelPath) override;
    size_t stateSize() const override;
    size_t saveState(uint8_t *dest) const override;
    size_t restoreState(const uint8_t *src) override;
    void setThreadCount(int32_t n_threads) override;
    int32_t threadCount() const override;

private:
    GPTJPrivate *d_ptr;

protected:
    std::vector<Token> tokenize(PromptContext &, const std::string&) const override;
    Token sampleToken(PromptContext &ctx) const override;
    std::string tokenToString(Token) const override;
    bool evalTokens(PromptContext &ctx, const std::vector<int32_t> &tokens) const override;
    int32_t contextLength() const override;
    const std::vector<Token>& endTokens() const override;
};

#endif // GPTJ_H
Dlopen backend 5 (#779) Major change to the backend that allows for pluggable versions of llama.cpp/ggml. This was squashed merged from dlopen_backend_5 where the history is preserved. 2023-05-31 21:04:01 +00:00			`#ifndef GPTJ_H_I_KNOW_WHAT_I_AM_DOING_WHEN_INCLUDING_THIS_FILE`
			`#error This file is NOT meant to be included outside of gptj.cpp. Doing so is DANGEROUS. Be sure to know what you are doing before proceeding to #define GPTJ_H_I_KNOW_WHAT_I_AM_DOING_WHEN_INCLUDING_THIS_FILE`
			`#endif`
Initial commit. 2023-04-09 03:28:39 +00:00			`#ifndef GPTJ_H`
			`#define GPTJ_H`

			`#include <string>`
			`#include <functional>`
Compile fix for MacOSX. 2023-04-10 19:04:40 +00:00			`#include <vector>`
Add an abstraction around gpt-j that will allow other arch models to be loaded in ui. 2023-04-14 02:15:40 +00:00			`#include "llmodel.h"`
Initial commit. 2023-04-09 03:28:39 +00:00
Dlopen backend 5 (#779) Major change to the backend that allows for pluggable versions of llama.cpp/ggml. This was squashed merged from dlopen_backend_5 where the history is preserved. 2023-05-31 21:04:01 +00:00			`struct GPTJPrivate;`
Add an abstraction around gpt-j that will allow other arch models to be loaded in ui. 2023-04-14 02:15:40 +00:00			`class GPTJ : public LLModel {`
Initial commit. 2023-04-09 03:28:39 +00:00			`public:`
			`GPTJ();`
			`~GPTJ();`

Add llama.cpp support for loading llama based models in the gui. We now support loading both gptj derived models and llama derived models. 2023-04-15 19:57:32 +00:00			`bool loadModel(const std::string &modelPath) override;`
Add an abstraction around gpt-j that will allow other arch models to be loaded in ui. 2023-04-14 02:15:40 +00:00			`bool isModelLoaded() const override;`
add requiredMem method to llmodel impls most of these can just shortcut out of the model loading logic llama is a bit worse to deal with because we submodule it so I have to at least parse the hparams, and then I just use the size on disk as an estimate for the mem size (which seems reasonable since we mmap() the llama files anyway) 2023-06-26 19:17:34 +00:00			`size_t requiredMem(const std::string &modelPath) override;`
Persistent state for gpt-j models too. 2023-05-05 14:00:05 +00:00			`size_t stateSize() const override;`
			`size_t saveState(uint8_t *dest) const override;`
			`size_t restoreState(const uint8_t *src) override;`
Add thread count setting 2023-04-18 13:46:03 +00:00			`void setThreadCount(int32_t n_threads) override;`
llmodel: constify LLModel::threadCount() 2023-05-21 20:45:29 +00:00			`int32_t threadCount() const override;`
Initial commit. 2023-04-09 03:28:39 +00:00
			`private:`
			`GPTJPrivate *d_ptr;`
Backend prompt dedup (#822) * Deduplicated prompt() function code 2023-06-04 12:59:24 +00:00
			`protected:`
Fix up for newer models on reset context. This fixes the model from totally failing after a reset context. 2023-06-04 23:31:00 +00:00			`std::vector<Token> tokenize(PromptContext &, const std::string&) const override;`
Backend prompt dedup (#822) * Deduplicated prompt() function code 2023-06-04 12:59:24 +00:00			`Token sampleToken(PromptContext &ctx) const override;`
llmodel: change tokenToString to not use string_view (#968) fixes a definite use-after-free and likely avoids some other potential ones - std::string will convert to a std::string_view automatically but as soon as the std::string in question goes out of scope it is already freed and the string_view is pointing at freed memory - this is mostly fine if its returning a reference to the tokenizer's internal vocab table but it's, imo, too easy to return a reference to a dynamically constructed string with this as replit is doing (and unfortunately needs to do to convert the internal whitespace replacement symbol back to a space) 2023-06-13 11:14:02 +00:00			`std::string tokenToString(Token) const override;`
Backend prompt dedup (#822) * Deduplicated prompt() function code 2023-06-04 12:59:24 +00:00			`bool evalTokens(PromptContext &ctx, const std::vector<int32_t> &tokens) const override;`
			`int32_t contextLength() const override;`
			`const std::vector<Token>& endTokens() const override;`
Initial commit. 2023-04-09 03:28:39 +00:00			`};`

Add thread count setting 2023-04-18 13:46:03 +00:00			`#endif // GPTJ_H`