Readme update.
This commit is contained in:
parent
2eba340523
commit
32ba623b96
31
readme.md
31
readme.md
|
@ -11,6 +11,8 @@ So far:
|
|||
of concept/springboard for more advanced narrative memory.
|
||||
- **GPU Scaling Filter:** Reduce number of GPU layers in use if Ollama
|
||||
crashes due to running out of VRAM.
|
||||
- **Output Sanitization Filter:** Remove words, phrases, or
|
||||
characters from the start of model replies.
|
||||
|
||||
## Memory Filter
|
||||
|
||||
|
@ -95,7 +97,7 @@ Other things to do or improve:
|
|||
events, instead of dumping it all into the vector DB.
|
||||
- Improve multi-user handling. Should technically sort of work due to
|
||||
messages having UUIDs, but is a bit messy. Only one collection is
|
||||
used, so multiple users = concurrency issues.
|
||||
used, so multiple users = concurrency issues?
|
||||
- Block user input while updating the knowledgebase.
|
||||
|
||||
## GPU Scaling Filter
|
||||
|
@ -108,6 +110,7 @@ account the number of layers in models or dynamically monitor VRAM
|
|||
use.
|
||||
|
||||
There are three settings:
|
||||
|
||||
- **Initial Reduction:** Number of layers to immediately set when an
|
||||
Ollama crash is detected. Defaults to 20.
|
||||
- **Scaling Step:** Number of layers to reduce by on subsequent crashes
|
||||
|
@ -115,6 +118,30 @@ There are three settings:
|
|||
- **Show Status:** Whether or not to inform the user that the
|
||||
conversation is running slower due to GPU layer downscaling.
|
||||
|
||||
## Output Sanitization Filter
|
||||
|
||||
This filter is intended for models that often output unwanted
|
||||
characters or terms at the beginning of replies. I have noticed this
|
||||
especially with Beyonder V3 and related models. They sometimes output
|
||||
a `":"` or `"Name:"` in front of replies. For example, if system prompt is
|
||||
`"You are Quinn, a helpful assistant."` the model will often reply with
|
||||
`"Quinn:"` as its first word.
|
||||
|
||||
There is one setting:
|
||||
|
||||
- **Terms:** List of terms or characters to remove. This is a list,
|
||||
and in the UI, each item should be separated by a comma.
|
||||
|
||||
For the above example, the setting textbox should have `:,Quinn:` in
|
||||
it, to remove a single colon from the start of replies, and `Quinn:`
|
||||
from the start of replies.
|
||||
|
||||
### Other Notes
|
||||
|
||||
Terms are removed in the order defined by the setting. The filter
|
||||
loops through each term and attempts to remove it from the start of
|
||||
the LLM's reply.
|
||||
|
||||
# License
|
||||
|
||||
AGPL v3.0+.
|
||||
All filters are licensed under AGPL v3.0+.
|
||||
|
|
Loading…
Reference in New Issue