Readme update.
This commit is contained in:
parent
2eba340523
commit
0a6974bf2a
Binary file not shown.
After Width: | Height: | Size: 3.5 KiB |
41
readme.md
41
readme.md
|
@ -11,6 +11,8 @@ So far:
|
||||||
of concept/springboard for more advanced narrative memory.
|
of concept/springboard for more advanced narrative memory.
|
||||||
- **GPU Scaling Filter:** Reduce number of GPU layers in use if Ollama
|
- **GPU Scaling Filter:** Reduce number of GPU layers in use if Ollama
|
||||||
crashes due to running out of VRAM.
|
crashes due to running out of VRAM.
|
||||||
|
- **Output Sanitization Filter:** Remove words, phrases, or
|
||||||
|
characters from the start of model replies.
|
||||||
|
|
||||||
## Memory Filter
|
## Memory Filter
|
||||||
|
|
||||||
|
@ -95,7 +97,7 @@ Other things to do or improve:
|
||||||
events, instead of dumping it all into the vector DB.
|
events, instead of dumping it all into the vector DB.
|
||||||
- Improve multi-user handling. Should technically sort of work due to
|
- Improve multi-user handling. Should technically sort of work due to
|
||||||
messages having UUIDs, but is a bit messy. Only one collection is
|
messages having UUIDs, but is a bit messy. Only one collection is
|
||||||
used, so multiple users = concurrency issues.
|
used, so multiple users = concurrency issues?
|
||||||
- Block user input while updating the knowledgebase.
|
- Block user input while updating the knowledgebase.
|
||||||
|
|
||||||
## GPU Scaling Filter
|
## GPU Scaling Filter
|
||||||
|
@ -108,6 +110,7 @@ account the number of layers in models or dynamically monitor VRAM
|
||||||
use.
|
use.
|
||||||
|
|
||||||
There are three settings:
|
There are three settings:
|
||||||
|
|
||||||
- **Initial Reduction:** Number of layers to immediately set when an
|
- **Initial Reduction:** Number of layers to immediately set when an
|
||||||
Ollama crash is detected. Defaults to 20.
|
Ollama crash is detected. Defaults to 20.
|
||||||
- **Scaling Step:** Number of layers to reduce by on subsequent crashes
|
- **Scaling Step:** Number of layers to reduce by on subsequent crashes
|
||||||
|
@ -115,6 +118,40 @@ There are three settings:
|
||||||
- **Show Status:** Whether or not to inform the user that the
|
- **Show Status:** Whether or not to inform the user that the
|
||||||
conversation is running slower due to GPU layer downscaling.
|
conversation is running slower due to GPU layer downscaling.
|
||||||
|
|
||||||
|
## Output Sanitization Filter
|
||||||
|
|
||||||
|
This filter is intended for models that often output unwanted
|
||||||
|
characters or terms at the beginning of replies. I have noticed this
|
||||||
|
especially with Beyonder V3 and related models. They sometimes output
|
||||||
|
a `":"` or `"Name:"` in front of replies. For example, if system prompt is
|
||||||
|
`"You are Quinn, a helpful assistant."` the model will often reply with
|
||||||
|
`"Quinn:"` as its first word.
|
||||||
|
|
||||||
|
There is one setting:
|
||||||
|
|
||||||
|
- **Terms:** List of terms or characters to remove. This is a list,
|
||||||
|
and in the UI, each item should be separated by a comma.
|
||||||
|
|
||||||
|
For the above example, the setting textbox should have `:,Quinn:` in
|
||||||
|
it, to remove a single colon from the start of replies, and `Quinn:`
|
||||||
|
from the start of replies.
|
||||||
|
|
||||||
|
### Other Notes
|
||||||
|
|
||||||
|
Terms are removed in the order defined by the setting. The filter
|
||||||
|
loops through each term and attempts to remove it from the start of
|
||||||
|
the LLM's reply.
|
||||||
|
|
||||||
# License
|
# License
|
||||||
|
|
||||||
AGPL v3.0+.
|
<img src="./agplv3.png" alt="AGPLv3" />
|
||||||
|
|
||||||
|
All filters are licensed under [AGPL v3.0+][agpl]. The code is free
|
||||||
|
software, that you can run, redistribute, modify, study, and learn
|
||||||
|
from as you see fit, as long as you extend that same freedom to
|
||||||
|
others, in accordance with the terms of the AGPL.
|
||||||
|
|
||||||
|
Be aware how this might affect your OpenWebUI deployment, if you are
|
||||||
|
deploying OpenWebUI in a public environment.
|
||||||
|
|
||||||
|
[agpl]: https://www.gnu.org/licenses/agpl-3.0.en.html
|
||||||
|
|
Loading…
Reference in New Issue