Initial commit

This commit is contained in:
ItzCrazyKns 2024-04-09 16:21:05 +05:30
commit d1c74c861e
No known key found for this signature in database
GPG Key ID: 8162927C7CCE3065
57 changed files with 4568 additions and 0 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 151 KiB

4
.env.example Normal file
View File

@ -0,0 +1,4 @@
PORT=3001
OPENAI_API_KEY=
SIMILARITY_MEASURE=cosine # cosine or dot
SEARXNG_API_URL= # no need to fill this if using docker

35
.gitignore vendored Normal file
View File

@ -0,0 +1,35 @@
# Node.js
node_modules/
npm-debug.log
yarn-error.log
# Build output
/.next/
/out/
# IDE/Editor specific
.vscode/
.idea/
*.iml
# Dependency lock files
package-lock.json
yarn.lock
# Environment variables
.env
.env.local
.env.development.local
.env.test.local
.env.production.local
# Log files
logs/
*.log
# Testing
/coverage/
# Miscellaneous
.DS_Store
Thumbs.db

12
.prettierrc.js Normal file
View File

@ -0,0 +1,12 @@
/** @type {import("prettier").Config} */
const config = {
printWidth: 80,
trailingComma: 'all',
endOfLine: 'auto',
singleQuote: true,
tabWidth: 2,
semi: true,
};
module.exports = config;

74
README.md Normal file
View File

@ -0,0 +1,74 @@
# 🚀 Perplexica - An AI-powered search engine 🔎
![preview](.assets/perplexica-screenshot.png)
## Overview
Perplexica is an open-source AI-powered searching tool or an AI-powered search engine that goes deep into the internet to find answers. Inspired by Perplexity AI, it's an open-source option that not just searches the web but understands your questions. It uses advanced machine learning algorithms like similarity searching and embeddings to refine results and provides clear answers with sources cited.
## Preview
![video-preview](.assets/perplexica-preview.gif)
## Features
- **Two Main Modes:**
- **Copilot Mode:** (In development) Boosts search by generating different queries to find more relevant internet sources. Like normal search instead of just using the context by SearxNG, it visits the top matches and tries to find relevant sources to the user's query directly from the page.
- **Normal Mode:** Processes your query and performs a web search.
- **Focus Modes:** (In development) special modes to better answer specific types of questions.
- **Current Information:** Some search tools might give you outdated info because they use data from crawling bots and convert them into embeddings and store them in a index (its like converting the web into embeddings which is quite expensive.). Unlike them, Perplexica uses SearxNG, a metasearch engine to get the results and rerank and get the most relevent source out of it, ensuring you always get the latest information without the overhead of daily data updates.
It has many more features like image and video search. Some of the planned features are mentioned in [upcoming features](#upcoming-features).
## Installation
There are mainly 2 ways of installing Perplexica - With Docker, Without Docker. Using Docker is highly recommended.
### Getting Started with Docker (Recommended)
1. Make sure Docker is installed and running on your system.
2. Clone the Perplexica repository:
```bash
git clone https://github.com/ItzCrazyKns/Perplexica.git
```
3. After cloning, rename the `.env.example` file to `.env` in the root directory. For Docker setups, you only need to fill these fields:
- `OPENAI_API_KEY`
- `SIMILARITY_MEASURE` (Its filled by default, you can leave it if you do not know about it.)
4. Navigate to the directory containing `docker-compose.yaml` and execute:
```bash
docker compose up
```
5. Wait a few minutes for the setup to complete. Access Perplexica at `http://localhost:3001` in your web browser.
### Non-Docker Installation
For setups without Docker:
1. Follow the initial steps to clone the repository and rename the `.env.example` file to `.env` in the root directory. You will need to fill in all the fields in this file.
2. Additionally, rename the `.env.example` file to `.env` in the `ui` folder and complete all fields.
3. The non-Docker setup requires manual configuration of both the backend and frontend.
**Note**: Using Docker is recommended as it simplifies the setup process, especially for managing environment variables and dependencies.
## Upcoming Features
- Finalizing Copilot Mode
- Adding support for multiple local LLMs and LLM providers such as Anthropic, Google, etc.
- Adding Discover and History Saving features
- Introducing various Focus Modes
- Continuous bug fixing
## Contribution
Perplexica is built on the idea that AI and large language models should be easy for everyone to use. If you find bugs or have ideas, please share them in via GitHub Issues. Details on how to contribute will be shared soon.
## Acknowledgements
Inspired by Perplexity AI, Perplexica aims to provide a similar service but always up-to-date and fully open source, thanks to SearxNG.
If you have any queries you can reach me via my Discord - `itzcrazykns`. Thanks for checking out Perplexica.

15
app.dockerfile Normal file
View File

@ -0,0 +1,15 @@
FROM node:alpine
ARG NEXT_PUBLIC_WS_URL
ARG NEXT_PUBLIC_API_URL
ENV NEXT_PUBLIC_WS_URL=${NEXT_PUBLIC_WS_URL}
ENV NEXT_PUBLIC_API_URL=${NEXT_PUBLIC_API_URL}
WORKDIR /home/perplexica
COPY ui /home/perplexica/
RUN yarn install
RUN yarn build
CMD ["yarn", "start"]

17
backend.dockerfile Normal file
View File

@ -0,0 +1,17 @@
FROM node:alpine
ARG SEARXNG_API_URL
ENV SEARXNG_API_URL=${SEARXNG_API_URL}
WORKDIR /home/perplexica
COPY src /home/perplexica/src
COPY tsconfig.json /home/perplexica/
COPY .env /home/perplexica/
COPY package.json /home/perplexica/
COPY yarn.lock /home/perplexica/
RUN yarn install
RUN yarn build
CMD ["yarn", "start"]

44
docker-compose.yaml Normal file
View File

@ -0,0 +1,44 @@
services:
searxng:
build:
context: .
dockerfile: searxng.dockerfile
expose:
- 4000
ports:
- 4000:8080
networks:
- perplexica-network
perplexica-backend:
build:
context: .
dockerfile: backend.dockerfile
args:
- SEARXNG_API_URL=http://searxng:8080
depends_on:
- searxng
expose:
- 3001
ports:
- 3001:3001
networks:
- perplexica-network
perplexica-frontend:
build:
context: .
dockerfile: app.dockerfile
args:
- NEXT_PUBLIC_API_URL=http://127.0.0.1:3001/api
- NEXT_PUBLIC_WS_URL=ws://127.0.0.1:3001
depends_on:
- perplexica-backend
expose:
- 3000
ports:
- 3000:3000
networks:
- perplexica-network
networks:
perplexica-network:

32
package.json Normal file
View File

@ -0,0 +1,32 @@
{
"name": "perplexica-backend",
"version": "1.0.0",
"license": "MIT",
"author": "ItzCrazyKns",
"scripts": {
"start": "node --env-file=.env dist/app.js",
"build": "tsc",
"dev": "nodemon --env-file=.env src/app.ts",
"format": "prettier . --check",
"format:write": "prettier . --write"
},
"devDependencies": {
"@types/cors": "^2.8.17",
"@types/express": "^4.17.21",
"@types/readable-stream": "^4.0.11",
"prettier": "^3.2.5",
"ts-node": "^10.9.2",
"typescript": "^5.4.3"
},
"dependencies": {
"@langchain/openai": "^0.0.25",
"axios": "^1.6.8",
"compute-cosine-similarity": "^1.1.0",
"compute-dot": "^1.1.0",
"cors": "^2.8.5",
"express": "^4.19.2",
"langchain": "^0.1.30",
"ws": "^8.16.0",
"zod": "^3.22.4"
}
}

2380
searxng-settings.yml Normal file

File diff suppressed because it is too large Load Diff

3
searxng.dockerfile Normal file
View File

@ -0,0 +1,3 @@
FROM searxng/searxng
COPY searxng-settings.yml /etc/searxng/settings.yml

View File

@ -0,0 +1,80 @@
import {
RunnableSequence,
RunnableMap,
RunnableLambda,
} from '@langchain/core/runnables';
import { PromptTemplate } from '@langchain/core/prompts';
import { OpenAI } from '@langchain/openai';
import formatChatHistoryAsString from '../utils/formatHistory';
import { BaseMessage } from '@langchain/core/messages';
import { StringOutputParser } from '@langchain/core/output_parsers';
import { searchSearxng } from '../core/searxng';
const llm = new OpenAI({
temperature: 0,
modelName: 'gpt-3.5-turbo',
});
const imageSearchChainPrompt = `
You will be given a conversation below and a follow up question. You need to rephrase the follow-up question so it is a standalone question that can be used by the LLM to search the web for images.
You need to make sure the rephrased question agrees with the conversation and is relevant to the conversation.
Example:
1. Follow up question: What is a cat?
Rephrased: A cat
2. Follow up question: What is a car? How does it works?
Rephrased: Car working
3. Follow up question: How does an AC work?
Rephrased: AC working
Conversation:
{chat_history}
Follow up question: {query}
Rephrased question:
`;
type ImageSearchChainInput = {
chat_history: BaseMessage[];
query: string;
};
const strParser = new StringOutputParser();
const imageSearchChain = RunnableSequence.from([
RunnableMap.from({
chat_history: (input: ImageSearchChainInput) => {
return formatChatHistoryAsString(input.chat_history);
},
query: (input: ImageSearchChainInput) => {
return input.query;
},
}),
PromptTemplate.fromTemplate(imageSearchChainPrompt),
llm,
strParser,
RunnableLambda.from(async (input: string) => {
const res = await searchSearxng(input, {
categories: ['images'],
engines: ['bing_images', 'google_images'],
});
const images = [];
res.results.forEach((result) => {
if (result.img_src && result.url && result.title) {
images.push({
img_src: result.img_src,
url: result.url,
title: result.title,
});
}
});
return images.slice(0, 10);
}),
]);
export default imageSearchChain;

View File

@ -0,0 +1,250 @@
import { BaseMessage } from '@langchain/core/messages';
import {
PromptTemplate,
ChatPromptTemplate,
MessagesPlaceholder,
} from '@langchain/core/prompts';
import {
RunnableSequence,
RunnableMap,
RunnableLambda,
} from '@langchain/core/runnables';
import { ChatOpenAI, OpenAI, OpenAIEmbeddings } from '@langchain/openai';
import { StringOutputParser } from '@langchain/core/output_parsers';
import { Document } from '@langchain/core/documents';
import { searchSearxng } from '../core/searxng';
import type { StreamEvent } from '@langchain/core/tracers/log_stream';
import formatChatHistoryAsString from '../utils/formatHistory';
import eventEmitter from 'events';
import computeSimilarity from '../utils/computeSimilarity';
const chatLLM = new ChatOpenAI({
modelName: 'gpt-3.5-turbo',
temperature: 0.7,
});
const llm = new OpenAI({
temperature: 0,
modelName: 'gpt-3.5-turbo',
});
const embeddings = new OpenAIEmbeddings({
modelName: 'text-embedding-3-large',
});
const basicSearchRetrieverPrompt = `
You will be given a conversation below and a follow up question. You need to rephrase the follow-up question if needed so it is a standalone question that can be used by the LLM to search the web for information.
If it is a writing task or a simple hi, hello rather than a question, you need to return \`not_needed\` as the response.
Example:
1. Follow up question: What is the capital of France?
Rephrased: Capital of france
2. Follow up question: What is the population of New York City?
Rephrased: Population of New York City
3. Follow up question: What is Docker?
Rephrased: What is Docker
Conversation:
{chat_history}
Follow up question: {query}
Rephrased question:
`;
const basicWebSearchResponsePrompt = `
You are Perplexica, an AI model who is expert at searching the web and answering user's queries.
Generate a response that is informative and relevant to the user's query based on provided context (the context consits of search results containg a brief description of the content of that page).
You must use this context to answer the user's query in the best way possible. Use an unbaised and journalistic tone in your response. Do not repeat the text.
You must not tell the user to open any link or visit any website to get the answer. You must provide the answer in the response itself. If the user asks for links you can provide them.
Your responses should be medium to long in length be informative and relevant to the user's query. You can use markdowns to format your response. You should use bullet points to list the information. Make sure the answer is not short and is informative.
You have to cite the answer using [number] notation. You must cite the sentences with their relevent context number. You must cite each and every part of the answer so the user can know where the information is coming from.
Place these citations at the end of that particular sentence. You can cite the same sentence multiple times if it is relevant to the user's query like [number1][number2].
However you do not need to cite it using the same number. You can use different numbers to cite the same sentence multiple times. The number refers to the number of the search result (passed in the context) used to generate that part of the answer.
Aything inside the following \`context\` HTML block provided below is for your knowledge returned by the search engine and is not shared by the user. You have to answer question on the basis of it and cite the relevant information from it but you do not have to
talk about the context in your response.
<context>
{context}
</context>
If you think there's nothing relevant in the search results, you can say that 'Hmm, sorry I could not find any relevant information on this topic. Would you like me to search again or ask something else?'.
Anything between the \`context\` is retrieved from a search engine and is not a part of the conversation with the user. Today's date is ${new Date().toISOString()}
`;
const strParser = new StringOutputParser();
const handleStream = async (
stream: AsyncGenerator<StreamEvent, any, unknown>,
emitter: eventEmitter,
) => {
for await (const event of stream) {
if (
event.event === 'on_chain_end' &&
event.name === 'FinalSourceRetriever'
) {
emitter.emit(
'data',
JSON.stringify({ type: 'sources', data: event.data.output }),
);
}
if (
event.event === 'on_chain_stream' &&
event.name === 'FinalResponseGenerator'
) {
emitter.emit(
'data',
JSON.stringify({ type: 'response', data: event.data.chunk }),
);
}
if (
event.event === 'on_chain_end' &&
event.name === 'FinalResponseGenerator'
) {
emitter.emit('end');
}
}
};
const processDocs = async (docs: Document[]) => {
return docs
.map((_, index) => `${index + 1}. ${docs[index].pageContent}`)
.join('\n');
};
const rerankDocs = async ({
query,
docs,
}: {
query: string;
docs: Document[];
}) => {
if (docs.length === 0) {
return docs;
}
const docsWithContent = docs.filter(
(doc) => doc.pageContent && doc.pageContent.length > 0,
);
const docEmbeddings = await embeddings.embedDocuments(
docsWithContent.map((doc) => doc.pageContent),
);
const queryEmbedding = await embeddings.embedQuery(query);
const similarity = docEmbeddings.map((docEmbedding, i) => {
const sim = computeSimilarity(queryEmbedding, docEmbedding);
return {
index: i,
similarity: sim,
};
});
const sortedDocs = similarity
.sort((a, b) => b.similarity - a.similarity)
.filter((sim) => sim.similarity > 0.5)
.slice(0, 15)
.map((sim) => docsWithContent[sim.index]);
return sortedDocs;
};
type BasicChainInput = {
chat_history: BaseMessage[];
query: string;
};
const basicWebSearchRetrieverChain = RunnableSequence.from([
PromptTemplate.fromTemplate(basicSearchRetrieverPrompt),
llm,
strParser,
RunnableLambda.from(async (input: string) => {
if (input === 'not_needed') {
return { query: '', docs: [] };
}
const res = await searchSearxng(input, {
language: 'en',
});
const documents = res.results.map(
(result) =>
new Document({
pageContent: result.content,
metadata: {
title: result.title,
url: result.url,
...(result.img_src && { img_src: result.img_src }),
},
}),
);
return { query: input, docs: documents };
}),
]);
const basicWebSearchAnsweringChain = RunnableSequence.from([
RunnableMap.from({
query: (input: BasicChainInput) => input.query,
chat_history: (input: BasicChainInput) => input.chat_history,
context: RunnableSequence.from([
(input) => ({
query: input.query,
chat_history: formatChatHistoryAsString(input.chat_history),
}),
basicWebSearchRetrieverChain
.pipe(rerankDocs)
.withConfig({
runName: 'FinalSourceRetriever',
})
.pipe(processDocs),
]),
}),
ChatPromptTemplate.fromMessages([
['system', basicWebSearchResponsePrompt],
new MessagesPlaceholder('chat_history'),
['user', '{query}'],
]),
chatLLM,
strParser,
]).withConfig({
runName: 'FinalResponseGenerator',
});
const basicWebSearch = (query: string, history: BaseMessage[]) => {
const emitter = new eventEmitter();
try {
const stream = basicWebSearchAnsweringChain.streamEvents(
{
chat_history: history,
query: query,
},
{
version: 'v1',
},
);
handleStream(stream, emitter);
} catch (err) {
emitter.emit(
'error',
JSON.stringify({ data: 'An error has occurred please try again later' }),
);
console.error(err);
}
return emitter;
};
const handleWebSearch = (message: string, history: BaseMessage[]) => {
const emitter = basicWebSearch(message, history);
return emitter;
};
export default handleWebSearch;

26
src/app.ts Normal file
View File

@ -0,0 +1,26 @@
import { startWebSocketServer } from './websocket';
import express from 'express';
import cors from 'cors';
import http from 'http';
import routes from './routes';
const app = express();
const server = http.createServer(app);
const corsOptions = {
origin: '*',
};
app.use(cors(corsOptions));
app.use(express.json());
app.use('/api', routes);
app.get('/api', (_, res) => {
res.status(200).json({ status: 'ok' });
});
server.listen(process.env.PORT!, () => {
console.log(`API server started on port ${process.env.PORT}`);
});
startWebSocketServer(server);

69
src/core/agentPicker.ts Normal file
View File

@ -0,0 +1,69 @@
import { z } from 'zod';
import { OpenAI } from '@langchain/openai';
import { RunnableSequence } from '@langchain/core/runnables';
import { StructuredOutputParser } from 'langchain/output_parsers';
import { PromptTemplate } from '@langchain/core/prompts';
const availableAgents = [
{
name: 'webSearch',
description:
'It is expert is searching the web for information and answer user queries',
},
/* {
name: 'academicSearch',
description:
'It is expert is searching the academic databases for information and answer user queries. It is particularly good at finding research papers and articles on topics like science, engineering, and technology. Use this instead of wolframAlphaSearch if the user query is not mathematical or scientific in nature',
},
{
name: 'youtubeSearch',
description:
'This model is expert at finding videos on youtube based on user queries',
},
{
name: 'wolframAlphaSearch',
description:
'This model is expert at finding answers to mathematical and scientific questions based on user queries.',
},
{
name: 'redditSearch',
description:
'This model is expert at finding posts and discussions on reddit based on user queries',
},
{
name: 'writingAssistant',
description:
'If there is no need for searching, this model is expert at generating text based on user queries',
}, */
];
const parser = StructuredOutputParser.fromZodSchema(
z.object({
agent: z.string().describe('The name of the selected agent'),
}),
);
const prompt = `
You are an AI model who is expert at finding suitable agents for user queries. The available agents are:
${availableAgents.map((agent) => `- ${agent.name}: ${agent.description}`).join('\n')}
Your task is to find the most suitable agent for the following query: {query}
{format_instructions}
`;
const chain = RunnableSequence.from([
PromptTemplate.fromTemplate(prompt),
new OpenAI({ temperature: 0 }),
parser,
]);
const pickSuitableAgent = async (query: string) => {
const res = await chain.invoke({
query,
format_instructions: parser.getFormatInstructions(),
});
return res.agent;
};
export default pickSuitableAgent;

42
src/core/searxng.ts Normal file
View File

@ -0,0 +1,42 @@
import axios from 'axios';
interface SearxngSearchOptions {
categories?: string[];
engines?: string[];
language?: string;
pageno?: number;
}
interface SearxngSearchResult {
title: string;
url: string;
img_src?: string;
thumbnail_src?: string;
content?: string;
author?: string;
}
export const searchSearxng = async (
query: string,
opts?: SearxngSearchOptions,
) => {
const url = new URL(`${process.env.SEARXNG_API_URL}/search?format=json`);
url.searchParams.append('q', query);
if (opts) {
Object.keys(opts).forEach((key) => {
if (Array.isArray(opts[key])) {
url.searchParams.append(key, opts[key].join(','));
return;
}
url.searchParams.append(key, opts[key]);
});
}
const res = await axios.get(url.toString());
const results: SearxngSearchResult[] = res.data.results;
const suggestions: string[] = res.data.suggestions;
return { results, suggestions };
};

22
src/routes/images.ts Normal file
View File

@ -0,0 +1,22 @@
import express from 'express';
import imageSearchChain from '../agents/imageSearchAgent';
const router = express.Router();
router.post('/', async (req, res) => {
try {
const { query, chat_history } = req.body;
const images = await imageSearchChain.invoke({
query,
chat_history,
});
res.status(200).json({ images });
} catch (err) {
res.status(500).json({ message: 'An error has occurred.' });
console.log(err.message);
}
});
export default router;

8
src/routes/index.ts Normal file
View File

@ -0,0 +1,8 @@
import express from 'express';
import imagesRouter from './images';
const router = express.Router();
router.use('/images', imagesRouter);
export default router;

View File

@ -0,0 +1,14 @@
import dot from 'compute-dot';
import cosineSimilarity from 'compute-cosine-similarity';
const computeSimilarity = (x: number[], y: number[]): number => {
if (process.env.SIMILARITY_MEASURE === 'cosine') {
return cosineSimilarity(x, y);
} else if (process.env.SIMILARITY_MEASURE === 'dot') {
return dot(x, y);
}
throw new Error('Invalid similarity measure');
};
export default computeSimilarity;

View File

@ -0,0 +1,9 @@
import { BaseMessage } from '@langchain/core/messages';
const formatChatHistoryAsString = (history: BaseMessage[]) => {
return history
.map((message) => `${message._getType()}: ${message.content}`)
.join('\n');
};
export default formatChatHistoryAsString;

View File

@ -0,0 +1,11 @@
import { WebSocket } from 'ws';
import { handleMessage } from './messageHandler';
export const handleConnection = (ws: WebSocket) => {
ws.on(
'message',
async (message) => await handleMessage(message.toString(), ws),
);
ws.on('close', () => console.log('Connection closed'));
};

8
src/websocket/index.ts Normal file
View File

@ -0,0 +1,8 @@
import { initServer } from './websocketServer';
import http from 'http';
export const startWebSocketServer = (
server: http.Server<typeof http.IncomingMessage, typeof http.ServerResponse>,
) => {
initServer(server);
};

View File

@ -0,0 +1,81 @@
import { WebSocket } from 'ws';
import pickSuitableAgent from '../core/agentPicker';
import handleWebSearch from '../agents/webSearchAgent';
import { BaseMessage, AIMessage, HumanMessage } from '@langchain/core/messages';
type Message = {
type: string;
content: string;
copilot: boolean;
focus: string;
history: Array<[string, string]>;
};
export const handleMessage = async (message: string, ws: WebSocket) => {
try {
const parsedMessage = JSON.parse(message) as Message;
const id = Math.random().toString(36).substring(7);
if (!parsedMessage.content)
return ws.send(
JSON.stringify({ type: 'error', data: 'Invalid message format' }),
);
const history: BaseMessage[] = parsedMessage.history.map((msg) => {
if (msg[0] === 'human') {
return new HumanMessage({
content: msg[1],
});
} else {
return new AIMessage({
content: msg[1],
});
}
});
if (parsedMessage.type === 'message') {
/* if (!parsedMessage.focus) {
const agent = await pickSuitableAgent(parsedMessage.content);
parsedMessage.focus = agent;
} */
parsedMessage.focus = 'webSearch';
switch (parsedMessage.focus) {
case 'webSearch': {
const emitter = handleWebSearch(parsedMessage.content, history);
emitter.on('data', (data) => {
const parsedData = JSON.parse(data);
if (parsedData.type === 'response') {
ws.send(
JSON.stringify({
type: 'message',
data: parsedData.data,
messageId: id,
}),
);
} else if (parsedData.type === 'sources') {
ws.send(
JSON.stringify({
type: 'sources',
data: parsedData.data,
messageId: id,
}),
);
}
});
emitter.on('end', () => {
ws.send(JSON.stringify({ type: 'messageEnd', messageId: id }));
});
emitter.on('error', (data) => {
const parsedData = JSON.parse(data);
ws.send(JSON.stringify({ type: 'error', data: parsedData.data }));
});
}
}
}
} catch (error) {
console.error('Failed to handle message', error);
ws.send(JSON.stringify({ type: 'error', data: 'Invalid message format' }));
}
};

View File

@ -0,0 +1,15 @@
import { WebSocketServer } from 'ws';
import { handleConnection } from './connectionManager';
import http from 'http';
export const initServer = (
server: http.Server<typeof http.IncomingMessage, typeof http.ServerResponse>,
) => {
const wss = new WebSocketServer({ server });
wss.on('connection', (ws) => {
handleConnection(ws);
});
console.log(`WebSocket server started on port ${process.env.PORT}`);
};

17
tsconfig.json Normal file
View File

@ -0,0 +1,17 @@
{
"compilerOptions": {
"lib": ["ESNext"],
"module": "commonjs",
"target": "ESNext",
"outDir": "dist",
"sourceMap": false,
"esModuleInterop": true,
"experimentalDecorators": true,
"emitDecoratorMetadata": true,
"allowSyntheticDefaultImports": true,
"skipLibCheck": true,
"skipDefaultLibCheck": true
},
"include": ["src"],
"exclude": ["node_modules", "**/*.spec.ts"]
}

2
ui/.env.example Normal file
View File

@ -0,0 +1,2 @@
NEXT_PUBLIC_WS_URL=ws://localhost:3001
NEXT_PUBLIC_API_URL=http://localhost:3001/api

3
ui/.eslintrc.json Normal file
View File

@ -0,0 +1,3 @@
{
"extends": "next/core-web-vitals"
}

34
ui/.gitignore vendored Normal file
View File

@ -0,0 +1,34 @@
# dependencies
/node_modules
/.pnp
.pnp.js
.yarn/install-state.gz
# testing
/coverage
# next.js
/.next/
/out/
# production
/build
# misc
.DS_Store
*.pem
# debug
npm-debug.log*
yarn-debug.log*
yarn-error.log*
# local env files
.env*.local
# vercel
.vercel
# typescript
*.tsbuildinfo
next-env.d.ts

11
ui/.prettierrc.js Normal file
View File

@ -0,0 +1,11 @@
/** @type {import("prettier").Config} */
const config = {
printWidth: 80,
trailingComma: 'all',
endOfLine: 'auto',
singleQuote: true,
tabWidth: 2,
};
module.exports = config;

BIN
ui/app/favicon.ico Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

13
ui/app/globals.css Normal file
View File

@ -0,0 +1,13 @@
@tailwind base;
@tailwind components;
@tailwind utilities;
@layer base {
.overflow-hidden-scrollable {
-ms-overflow-style: none;
}
.overflow-hidden-scrollable::-webkit-scrollbar {
display: none;
}
}

32
ui/app/layout.tsx Normal file
View File

@ -0,0 +1,32 @@
import type { Metadata } from 'next';
import { Montserrat } from 'next/font/google';
import './globals.css';
import { cn } from '@/lib/utils';
import Sidebar from '@/components/Sidebar';
const montserrat = Montserrat({
weight: ['300', '400', '500', '700'],
subsets: ['latin'],
display: 'swap',
fallback: ['Arial', 'sans-serif'],
});
export const metadata: Metadata = {
title: 'Perplexica - Chat with the internet',
description:
'Perplexica is an AI powered chatbot that is connected to the internet.',
};
export default function RootLayout({
children,
}: Readonly<{
children: React.ReactNode;
}>) {
return (
<html className="h-full" lang="en">
<body className={cn('h-full', montserrat.className)}>
<Sidebar>{children}</Sidebar>
</body>
</html>
);
}

17
ui/app/page.tsx Normal file
View File

@ -0,0 +1,17 @@
import ChatWindow from '@/components/ChatWindow';
import { Metadata } from 'next';
export const metadata: Metadata = {
title: 'Chat - Perplexica',
description: 'Chat with the internet, chat with Perplexica.',
};
const Home = () => {
return (
<div>
<ChatWindow />
</div>
);
};
export default Home;

87
ui/components/Chat.tsx Normal file
View File

@ -0,0 +1,87 @@
'use client';
import { useEffect, useRef, useState } from 'react';
import MessageInput from './MessageInput';
import { Message } from './ChatWindow';
import MessageBox from './MessageBox';
import MessageBoxLoading from './MessageBoxLoading';
const Chat = ({
loading,
messages,
sendMessage,
messageAppeared,
rewrite,
}: {
messages: Message[];
sendMessage: (message: string) => void;
loading: boolean;
messageAppeared: boolean;
rewrite: (messageId: string) => void;
}) => {
const [dividerWidth, setDividerWidth] = useState(0);
const dividerRef = useRef<HTMLDivElement | null>(null);
const messageEnd = useRef<HTMLDivElement | null>(null);
useEffect(() => {
const updateDividerWidth = () => {
if (dividerRef.current) {
setDividerWidth(dividerRef.current.scrollWidth);
}
};
updateDividerWidth();
window.addEventListener('resize', updateDividerWidth);
return () => {
window.removeEventListener('resize', updateDividerWidth);
};
});
useEffect(() => {
messageEnd.current?.scrollIntoView({ behavior: 'smooth' });
if (messages.length === 1) {
document.title = `${messages[0].content.substring(0, 30)} - Perplexica`;
}
}, [messages]);
return (
<div className="flex flex-col space-y-6 pt-8 pb-44 lg:pb-32 sm:mx-4 md:mx-8">
{messages.map((msg, i) => {
const isLast = i === messages.length - 1;
return (
<>
<MessageBox
key={i}
message={msg}
messageIndex={i}
history={messages}
loading={loading}
dividerRef={isLast ? dividerRef : undefined}
isLast={isLast}
rewrite={rewrite}
/>
{!isLast && msg.role === 'assistant' && (
<div className="h-px w-full bg-[#1C1C1C]" />
)}
</>
);
})}
{loading && !messageAppeared && <MessageBoxLoading />}
<div ref={messageEnd} className="h-0" />
{dividerWidth > 0 && (
<div
className="bottom-24 lg:bottom-10 fixed z-40"
style={{ width: dividerWidth }}
>
<MessageInput sendMessage={sendMessage} />
</div>
)}
</div>
);
};
export default Chat;

View File

@ -0,0 +1,170 @@
'use client';
import { useEffect, useState } from 'react';
import { Document } from '@langchain/core/documents';
import Navbar from './Navbar';
import Chat from './Chat';
import EmptyChat from './EmptyChat';
export type Message = {
id: string;
createdAt?: Date;
content: string;
role: 'user' | 'assistant';
sources?: Document[];
};
const useSocket = (url: string) => {
const [ws, setWs] = useState<WebSocket | null>(null);
useEffect(() => {
if (!ws) {
const ws = new WebSocket(url);
ws.onopen = () => {
console.log('[DEBUG] open');
setWs(ws);
};
}
return () => {
ws?.close();
console.log('[DEBUG] closed');
};
}, [ws, url]);
return ws;
};
const ChatWindow = () => {
const ws = useSocket(process.env.NEXT_PUBLIC_WS_URL!);
const [chatHistory, setChatHistory] = useState<[string, string][]>([]);
const [messages, setMessages] = useState<Message[]>([]);
const [loading, setLoading] = useState(false);
const [messageAppeared, setMessageAppeared] = useState(false);
const sendMessage = async (message: string) => {
if (loading) return;
setLoading(true);
setMessageAppeared(false);
let sources: Document[] | undefined = undefined;
let recievedMessage = '';
let added = false;
ws?.send(
JSON.stringify({
type: 'message',
content: message,
history: [...chatHistory, ['human', message]],
}),
);
setMessages((prevMessages) => [
...prevMessages,
{
content: message,
id: Math.random().toString(36).substring(7),
role: 'user',
},
]);
const messageHandler = (e: MessageEvent) => {
const data = JSON.parse(e.data);
if (data.type === 'sources') {
sources = data.data;
if (!added) {
setMessages((prevMessages) => [
...prevMessages,
{
content: '',
id: data.messageId,
role: 'assistant',
sources: sources,
},
]);
added = true;
}
setMessageAppeared(true);
}
if (data.type === 'message') {
if (!added) {
setMessages((prevMessages) => [
...prevMessages,
{
content: data.data,
id: data.messageId,
role: 'assistant',
sources: sources,
},
]);
added = true;
}
setMessages((prev) =>
prev.map((message) => {
if (message.id === data.messageId) {
return { ...message, content: message.content + data.data };
}
return message;
}),
);
recievedMessage += data.data;
setMessageAppeared(true);
}
if (data.type === 'messageEnd') {
setChatHistory((prevHistory) => [
...prevHistory,
['human', message],
['assistant', recievedMessage],
]);
ws?.removeEventListener('message', messageHandler);
setLoading(false);
}
};
ws?.addEventListener('message', messageHandler);
};
const rewrite = (messageId: string) => {
const index = messages.findIndex((msg) => msg.id === messageId);
if (index === -1) return;
const message = messages[index - 1];
setMessages((prev) => {
return [...prev.slice(0, messages.length > 2 ? index - 1 : 0)];
});
setChatHistory((prev) => {
return [...prev.slice(0, messages.length > 2 ? index - 1 : 0)];
});
sendMessage(message.content);
};
return (
<div>
{messages.length > 0 ? (
<>
<Navbar />
<Chat
loading={loading}
messages={messages}
sendMessage={sendMessage}
messageAppeared={messageAppeared}
rewrite={rewrite}
/>
</>
) : (
<EmptyChat sendMessage={sendMessage} />
)}
</div>
);
};
export default ChatWindow;

View File

@ -0,0 +1,18 @@
import EmptyChatMessageInput from './EmptyChatMessageInput';
const EmptyChat = ({
sendMessage,
}: {
sendMessage: (message: string) => void;
}) => {
return (
<div className="flex flex-col items-center justify-center min-h-screen max-w-screen-sm mx-auto p-2 space-y-8">
<h2 className="text-white/70 text-3xl font-medium -mt-8">
Research begins here.
</h2>
<EmptyChatMessageInput sendMessage={sendMessage} />
</div>
);
};
export default EmptyChat;

View File

@ -0,0 +1,61 @@
import { ArrowRight } from 'lucide-react';
import { useState } from 'react';
import TextareaAutosize from 'react-textarea-autosize';
import { Attach, CopilotToggle, Focus } from './MessageInputActions';
const EmptyChatMessageInput = ({
sendMessage,
}: {
sendMessage: (message: string) => void;
}) => {
const [copilotEnabled, setCopilotEnabled] = useState(false);
const [message, setMessage] = useState('');
return (
<form
onSubmit={(e) => {
e.preventDefault();
sendMessage(message);
setMessage('');
}}
onKeyDown={(e) => {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault();
sendMessage(message);
setMessage('');
}
}}
className="w-full"
>
<div className="flex flex-col bg-[#111111] px-5 pt-5 pb-2 rounded-lg w-full border border-[#1C1C1C]">
<TextareaAutosize
value={message}
onChange={(e) => setMessage(e.target.value)}
minRows={2}
className="bg-transparent placeholder:text-white/50 text-sm text-white resize-none focus:outline-none w-full max-h-24 lg:max-h-36 xl:max-h-48"
placeholder="Ask anything..."
/>
<div className="flex flex-row items-center justify-between mt-4">
<div className="flex flex-row items-center space-x-1 -mx-2">
<Focus />
<Attach />
</div>
<div className="flex flex-row items-center space-x-4 -mx-2">
<CopilotToggle
copilotEnabled={copilotEnabled}
setCopilotEnabled={setCopilotEnabled}
/>
<button
disabled={message.trim().length === 0}
className="bg-[#24A0ED] text-white disabled:text-white/50 hover:bg-opacity-85 transition duration-100 disabled:bg-[#ececec21] rounded-full p-2"
>
<ArrowRight className="bg-background" size={17} />
</button>
</div>
</div>
</div>
</form>
);
};
export default EmptyChatMessageInput;

9
ui/components/Layout.tsx Normal file
View File

@ -0,0 +1,9 @@
const Layout = ({ children }: { children: React.ReactNode }) => {
return (
<main className="lg:pl-20 bg-[#0A0A0A] min-h-screen">
<div className="max-w-screen-lg lg:mx-auto mx-4">{children}</div>
</main>
);
};
export default Layout;

View File

@ -0,0 +1,29 @@
import { Check, ClipboardList } from 'lucide-react';
import { Message } from '../ChatWindow';
import { useState } from 'react';
const Copy = ({
message,
initialMessage,
}: {
message: Message;
initialMessage: string;
}) => {
const [copied, setCopied] = useState(false);
return (
<button
onClick={() => {
const contentToCopy = `${initialMessage}${message.sources && message.sources.length > 0 && `\n\nCitations:\n${message.sources?.map((source: any, i: any) => `[${i + 1}] ${source.metadata.url}`).join(`\n`)}`}`;
navigator.clipboard.writeText(contentToCopy);