refactor: restructure codebase for v1 kasm docker release

This commit is contained in:
Kovasky Buezo
2026-07-06 21:19:16 -02:30
parent 8176fce0c7
commit 1c2b963f27
34 changed files with 7589 additions and 8447 deletions
+5
View File
@@ -0,0 +1,5 @@
node_modules
browser_profiles
.git
.env
*.tar
+3 -1
View File
@@ -6,4 +6,6 @@ test.js
test.mjs
config.js
config.mjs
browser_profiles
browser_profiles
*.log
local_copy_formatted_messages.txt
+45 -27
View File
@@ -1,37 +1,55 @@
# Use the official Node.js 20 image as the base image
FROM node:20
RUN chmod 1777 /tmp
# Install necessary dependencies for running Chrome
RUN apt-get update && apt-get install -y \
wget \
gnupg \
ca-certificates \
apt-transport-https \
xvfb \
&& rm -rf /var/lib/apt/lists/*
FROM kasmweb/chrome:1.19.0
# Install Google Chrome
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb [arch=amd64] https://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list \
&& apt-get update \
&& apt-get install -y google-chrome-stable \
&& rm -rf /var/lib/apt/lists/*
USER root
# Set up the working directory in the container
# Install Node.js 20.x
RUN rm -f /etc/apt/sources.list.d/google-chrome.list && \
apt-get update && \
apt-get install -y curl && \
curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && \
apt-get install -y nodejs && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Create app directory and set ownership to kasm-user
RUN mkdir -p /app && chown -R 1000:1000 /app
# Switch to kasm-user to run npm install safely
USER 1000
WORKDIR /app
# Copy package.json and package-lock.json to the working directory
COPY package*.json ./
# Copy application files (ignoring node_modules and browser_profiles)
COPY --chown=1000:1000 . /app
# Install Node.js dependencies
# Pre-create the browser_profiles and data directory so the volume inherits the correct ownership
RUN mkdir -p /app/browser_profiles /app/data && chown -R 1000:1000 /app/browser_profiles /app/data
# Install Node dependencies
RUN npm install
# Copy the rest of the application code
COPY . .
# Create a custom entrypoint script
USER root
RUN echo '#!/bin/bash\n\
# Start Kasm VNC server in the background\n\
/dockerstartup/vnc_startup.sh &\n\
\n\
# Wait for the X server to fully initialize\n\
sleep 5\n\
\n\
# Set environment variables for YOUChat_Proxy\n\
export USE_MANUAL_LOGIN="true"\n\
export HIDE_BROWSER="false"\n\
export BROWSER_PATH="/usr/bin/google-chrome"\n\
export DISPLAY=:1\n\
\n\
# Start the proxy server\n\
cd /app\n\
node src/index.mjs\n\
' > /app/start-proxy.sh && chmod +x /app/start-proxy.sh
# Expose the port your app runs on
EXPOSE 8080
USER 1000
# Command to run the application
# Expose Kasm VNC port and Proxy API port
EXPOSE 6901 8080
CMD [ "node", "index.mjs" ]
ENTRYPOINT ["/app/start-proxy.sh"]
+24 -28
View File
@@ -1,28 +1,24 @@
# 更新日志(2025年4月4日)
添加对gemini 2.5 pro的支持(与官方api相比不支持联网搜索)
---------------------------------------------------------
# Miaomiaomiao
A proxy for YOU Chat.
YOU.Com 转换为通用代理。
如果您觉得本项目对您有帮助,请考虑[请我喝杯蜜雪冰城](https://github.com/sponsors/Archeb?frequency=one-time)
If you find this project useful, please consider [buying me a cup of coffee](https://github.com/sponsors/Archeb?frequency=one-time);
[**Usage 使用方法**](usage.md)
**It is forbidden to use this project for profit.**
**仅供个人部署用于访问自己合法取得的订阅,严禁用于转售或其他商业用途。不提供任何技术支持、不为任何违规使用导致的封号负责。**
ASK THE ASSISTANT TO HELP YOU DEPLOY ↓
有部署问题请问部署助手: https://www.coze.com/store/bot/7383564439096131592?bot_id=true
您的电脑上必须安装有 Google Chrome 或者 Edge 浏览器才能够使用当前分支。旧版已不再维护。
Google Chrome must be installed on your system in order to use this proxy. Previous versions are no longer maintained.
# Changelog (April 4, 2025)
Added support for gemini 2.5 pro (does not support internet search compared to the official API)
---------------------------------------------------------
# Miaomiaomiao
A proxy for YOU Chat.
Convert YOU.Com into a generic proxy.
If you find this project useful, please consider [buying me a cup of coffee](https://github.com/sponsors/Archeb?frequency=one-time);
[**Usage**](usage.md)
**It is forbidden to use this project for profit.**
**This is for personal deployment only to access subscriptions you legally obtain. It is strictly prohibited for resale or other commercial purposes. No technical support is provided, and we are not responsible for any account bans caused by misuse.**
ASK THE ASSISTANT TO HELP YOU DEPLOY ↓
If you have deployment issues, ask the deployment assistant: https://www.coze.com/store/bot/7383564439096131592?bot_id=true
Google Chrome must be installed on your system in order to use this proxy. Previous versions are no longer maintained.
+6 -6
View File
@@ -1,7 +1,7 @@
export const config = {
"sessions": [
{
"cookie": `...`,
}
]
export const config = {
"sessions": [
{
"cookie": `...`,
}
]
}
+2
View File
@@ -0,0 +1,2 @@
*
!.gitignore
+22
View File
@@ -0,0 +1,22 @@
version: "3.8"
services:
chat-proxy:
build: .
container_name: youchat-proxy
ports:
- "8080:8080"
environment:
- NODE_ENV=production
- PORT=8080
- PASSWORD=${PASSWORD:-}
restart: unless-stopped
volumes:
- ./data:/app/data
- ./browser_profiles:/app/browser_profiles
- ./config.mjs:/app/config.mjs
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 5s
retries: 3
-262
View File
@@ -1,262 +0,0 @@
import {EventEmitter} from "events";
import {connect} from "puppeteer-real-browser";
import {v4 as uuidV4} from "uuid";
import path from "path";
import {fileURLToPath} from "url";
import {createDirectoryIfNotExists, sleep} from "../utils/cookieUtils.mjs";
import '../proxyAgent.mjs';
import NetworkMonitor from "../networkMonitor.mjs";
import {detectBrowser} from "../utils/browserDetector.mjs";
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
class HappyApiProvider {
constructor(config) {
this.config = config;
this.sessions = {};
this.preferredBrowser = 'auto';
this.networkMonitor = new NetworkMonitor();
}
async init() {
console.log(`本项目依赖 Chrome 或 Edge 浏览器,请勿关闭弹出的浏览器窗口。如果出现错误请检查是否已安装 Chrome 或 Edge 浏览器。`);
const browserPath = detectBrowser(this.preferredBrowser);
this.sessions = {};
// 手动登录
const currentUsername = 'manual_login';
createDirectoryIfNotExists(path.join(__dirname, "browser_profiles", currentUsername));
try {
const response = await connect({
headless: "auto",
turnstile: true,
customConfig: {
userDataDir: path.join(__dirname, "browser_profiles", currentUsername),
executablePath: browserPath,
},
});
const {page, browser} = response;
console.log(`请在打开的浏览器窗口中手动登录 happyapi.org`);
await page.goto("https://happyapi.org", {waitUntil: 'networkidle0'});
await sleep(3000); // 等待页面加载完毕
const sessionCookie = await this.waitForManualLogin(page);
if (sessionCookie) {
this.sessions[currentUsername] = {browser, page, sessionCookie, valid: true};
console.log(`成功获取会话 cookie`);
} else {
console.error(`未能检测到登录状态,请确保已成功登录`);
await browser.close();
}
} catch (error) {
console.error(`初始化浏览器失败`);
console.error(error);
}
// 开始网络监控
await this.networkMonitor.startMonitoring();
}
async waitForManualLogin(page) {
return new Promise((resolve) => {
const checkLoginStatus = async () => {
// 检查是否存在特定的登录成功标识元素
const isLoggedIn = await this.checkLoginStatus(page);
if (isLoggedIn) {
console.log(`检测到登录成功`);
const cookies = await page.cookies();
const sessionCookie = this.extractSessionCookie(cookies);
resolve(sessionCookie);
} else {
setTimeout(checkLoginStatus, 1000);
}
};
page.on('request', (request) => {
if (request.url() === 'https://happyapi.org/api/v1/auths/signin' && request.method() === 'POST') {
console.log('检测到登录请求');
checkLoginStatus();
}
});
checkLoginStatus();
});
}
extractSessionCookie(cookies) {
const cfClearance = cookies.find(c => c.name === 'cf_clearance')?.value;
const token = cookies.find(c => c.name === 'token')?.value;
if (cfClearance && token) {
return [
{
name: 'cf_clearance',
value: cfClearance,
domain: 'happyapi.org',
path: '/',
httpOnly: true,
secure: true,
sameSite: 'Lax',
},
{
name: 'token',
value: token,
domain: 'happyapi.org',
path: '/',
httpOnly: true,
secure: true,
sameSite: 'Lax',
}
];
} else {
console.error('无法提取有效的会话 cookie');
return null;
}
}
async checkLoginStatus(page) {
try {
const userNameElement = await page.waitForSelector('div.self-center.font-medium', {timeout: 5000});
const userName = await page.evaluate(element => element.textContent, userNameElement);
return true;
} catch {
return false;
}
}
async getCompletion({username, messages, stream = false, proxyModel}) {
const session = this.sessions[username];
if (!session || !session.valid) {
throw new Error(`用户 ${username} 的会话无效`);
}
//刷新页面
await session.page.goto("https://happyapi.org", {waitUntil: 'domcontentloaded'});
const {page} = session;
const emitter = new EventEmitter();
try {
let userQuery = messages.map(msg => msg.content).join("\n\n");
const inputSelector = 'textarea#chat-textarea';
const sendButtonSelector = 'button#send-message-button';
// 输入消息
await page.waitForSelector(inputSelector, {visible: true});
await page.evaluate((selector, message) => {
const textarea = document.querySelector(selector);
textarea.value = message;
// 手动触发事件
textarea.dispatchEvent(new Event('input', {bubbles: true}));
textarea.dispatchEvent(new Event('change', {bubbles: true}));
}, inputSelector, userQuery.trim());
await page.waitForTimeout(100);
// 点击发送按钮
await page.waitForSelector(sendButtonSelector, {visible: true});
await page.click(sendButtonSelector);
// 监听请求
const targetUrl = 'https://happyapi.org/api/chat/completions';
const traceId = uuidV4();
let finalResponse = ""; // 存储最终响应
let responseStarted = false; // 是否开始接收
let isEnding = false; // 是否正在结束
// 清理函数
const cleanup = async () => {
clearTimeout(responseTimeout);
page.removeListener('response', responseHandler);
};
const responseTimeout = setTimeout(() => {
if (!isEnding) {
isEnding = true;
emitter.emit('error', new Error('请求超时'));
cleanup();
}
}, 60000); // 60秒,可根据需要调整
const responseHandler = async (response) => {
if (response.url() === targetUrl) {
try {
const buffer = await response.buffer();
const text = buffer.toString('utf8');
// 处理流式数据
const lines = text.split('\n\n');
for (const line of lines) {
if (line.startsWith('data: ')) {
const data = line.substring(6).trim();
if (data === '[DONE]') {
if (!isEnding) {
isEnding = true;
console.log("请求结束");
emitter.emit('end');
cleanup();
}
return;
}
if (data.length === 0) {
continue;
}
const parsedData = JSON.parse(data);
const contentDelta = parsedData.choices[0].delta.content || '';
if (stream) {
emitter.emit('completion', traceId, contentDelta);
} else {
finalResponse += contentDelta;
}
}
}
} catch (e) {
if (!isEnding) {
isEnding = true;
console.error("请求发生错误", e);
emitter.emit('error', e);
cleanup();
}
}
}
};
// 监听response
page.on('response', responseHandler);
// 清除计时器和监听器
emitter.on('end', () => {
cleanup();
});
emitter.on('error', () => {
cleanup();
});
return {
completion: emitter,
cancel: () => {
}
};
} catch (error) {
emitter.emit('error', error);
return {
completion: emitter,
cancel: () => { }
};
}
}
}
export default HappyApiProvider;
-66
View File
@@ -1,66 +0,0 @@
const imageStorage = new Map();
let imageCounter = 0;
/**
* 保存图片到内存中
* @param {string} base64Data base64 数据
* @param {string} mediaType 媒体类型 ("image/jpeg")
* @returns {{ imageId: string, mediaType: string }} 返回图片信息
*/
export function storeImage(base64Data, mediaType) {
const randomPart = Math.floor(Math.random() * 10000);
const imageId = `image_${Date.now()}_${imageCounter++}_${randomPart}`;
imageStorage.set(imageId, { imageId, base64Data, mediaType });
// console.log(`Image stored with ID: ${imageId}, Media Type: ${mediaType}`);
// 打印存储的 base64
// console.log(`Base64 Data for Image ID ${imageId}: ${base64Data.substring(0, 100)}...`);
return { imageId, mediaType };
}
/**
* 获取图片数据
* @param {string} imageId 图片 ID
* @returns {object|null} base64 数据和媒体类型
*/
export function getImage(imageId) {
return imageStorage.get(imageId) || null;
}
/**
* 获取最后一个图片
* @returns {object|null} base64 数据和媒体类型
*/
export function getLastImage() {
const lastKey = Array.from(imageStorage.keys()).pop();
return lastKey ? imageStorage.get(lastKey) : null;
}
/**
* 删除图片数据
* @param {string} imageId 图片 ID
*/
export function removeImage(imageId) {
imageStorage.delete(imageId);
}
/**
* 清空所有存储的图片
*/
export function clearAllImages() {
imageStorage.clear();
imageCounter = 0;
// console.log("All images have been cleared from memory.");
}
/**
* 打印所有存储的图片
*/
export function printAllImages() {
console.log('Current images stored in memory:');
imageStorage.forEach((value, key) => {
console.log(`Image ID: ${key}, Media Type: ${value.mediaType}`);
console.log(`Base64 Data: ${value.base64Data.substring(0, 100)}...`);
});
}
+4 -14
View File
@@ -2,8 +2,9 @@
"name": "youchat-proxy",
"version": "1.0.0",
"description": "",
"main": "index.js",
"main": "src/index.mjs",
"scripts": {
"start": "node src/index.mjs",
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
@@ -11,27 +12,16 @@
"dependencies": {
"async-mutex": "^0.5.0",
"cookie": "^0.6.0",
"cookie-parser": "^1.4.7",
"crypto-js": "^4.2.0",
"dayjs": "^1.11.13",
"docx": "^8.5.0",
"express": "^4.19.2",
"form-data": "^4.0.0",
"geoip-lite": "^1.4.10",
"https-proxy-agent": "^7.0.5",
"localtunnel": "^2.0.2",
"ngrok": "^5.0.0-beta.2",
"node-fetch": "^3.3.2",
"puppeteer-core": "^24.4.0",
"puppeteer-real-browser": "^1.4.2",
"socket.io-client": "^4.8.0",
"socks-proxy-agent": "^8.0.4",
"unzipper": "^0.12.3",
"uuid": "^9.0.1",
"winston": "^3.17.0",
"ws": "^8.18.1"
},
"devDependencies": {
"playwright": "^1.51.0"
"uuid": "^9.0.1"
}
}
-8
View File
@@ -1,8 +0,0 @@
export const config = {
sessions: [
{
cookie: `...`
}
// 可以添加更多cookie
]
};
-468
View File
@@ -1,468 +0,0 @@
import { EventEmitter } from "events";
import { connect } from "puppeteer-real-browser";
import { v4 as uuidV4 } from "uuid";
import path from "path";
import { fileURLToPath } from "url";
import { createDirectoryIfNotExists, sleep, extractPerplexityCookie, getPerplexitySessionCookie } from "../utils/cookieUtils.mjs";
import '../proxyAgent.mjs';
import { detectBrowser } from '../utils/browserDetector.mjs';
import NetworkMonitor from '../networkMonitor.mjs';
import io from 'socket.io-client';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
class PerplexityProvider {
constructor(config) {
this.config = config;
this.sessions = {};
// 可以是 'chrome', 'edge', 或 'auto'
this.preferredBrowser = 'auto';
this.networkMonitor = new NetworkMonitor();
}
async init(config) {
console.log(`本项目依赖Chrome或Edge浏览器,请勿关闭弹出的浏览器窗口。如果出现错误请检查是否已安装Chrome或Edge浏览器。`);
const browserPath = detectBrowser(this.preferredBrowser); // 检测Chrome和Edge浏览器
this.sessions = {};
const timeout = 120000;
if (process.env.USE_MANUAL_LOGIN === "true") {
this.sessions['manual_login'] = {
configIndex: 0,
valid: false,
};
console.log("当前使用手动登录模式,跳过config.mjs文件中的 cookie 验证");
} else {
// 使用配置文件中的 cookie
for (let index = 0; index < config.sessions.length; index++) {
const session = config.sessions[index];
const extractedCookie = extractPerplexityCookie(session.cookie);
if (extractedCookie.sessionToken) {
const uniqueId = `user_${index}`;
this.sessions[uniqueId] = {
configIndex: index,
...extractedCookie,
valid: false,
};
console.log(`已添加 #${index} ${uniqueId} (Perplexity cookie)`);
} else {
console.error(`${index}个cookie无效,请重新获取。`);
console.error(`未检测到有效的__Secure-next-auth.session-token字段。`);
}
}
console.log(`已添加 ${Object.keys(this.sessions).length} 个 cookie,开始验证有效性`);
}
for (const originalUsername of Object.keys(this.sessions)) {
let currentUsername = originalUsername;
let session = this.sessions[currentUsername];
createDirectoryIfNotExists(path.join(__dirname, "browser_profiles", currentUsername));
try {
const response = await connect({
headless: "auto",
turnstile: true,
customConfig: {
userDataDir: path.join(__dirname, "browser_profiles", currentUsername),
executablePath: browserPath,
},
});
const {page, browser} = response;
if (process.env.USE_MANUAL_LOGIN === "true") {
console.log(`正在为 session #${session.configIndex} 进行手动登录...`);
await page.goto("https://www.perplexity.ai", {timeout: timeout});
await sleep(3000);
console.log(`请在打开的浏览器窗口中手动登录 Perplexity.ai (session #${session.configIndex})`);
const { sessionCookie, accountStatus } = await this.waitForManualLogin(page);
if (sessionCookie) {
const email = accountStatus.username || 'unknown_user';
this.sessions[email] = {
...session,
cookies: sessionCookie,
isPro: accountStatus.isPro,
};
delete this.sessions[currentUsername];
currentUsername = email;
session = this.sessions[currentUsername];
console.log(`成功获取 ${email} 登录的 cookie${accountStatus.isPro ? 'Pro 账号)' : ''}`);
} else {
console.error(`未能获取到 session #${session.configIndex} 有效登录的 cookie`);
await browser.close();
continue;
}
} else {
// 使用已有的 cookie
const perplexityCookies = getPerplexitySessionCookie(session);
await page.setCookie(...perplexityCookies);
await page.goto("https://www.perplexity.ai", {timeout: timeout});
await sleep(5000);
}
// 如果遇到 Cloudflare 挑战就多等一段时间
const pageContent = await page.content();
if (pageContent.indexOf("challenges.cloudflare.com") > -1) {
console.log(`请在30秒内完成人机验证 (${currentUsername})`);
await page.evaluate(() => {
alert("请在30秒内完成人机验证");
});
await sleep(30000);
}
try {
const isValid = await this.validateSession(page);
if (isValid) {
session.valid = true;
session.browser = browser;
session.page = page;
} else {
console.warn(`警告: ${currentUsername} 验证失败。请检查cookie是否有效。`);
await this.clearPerplexityCookies(page);
await browser.close();
}
} catch (e) {
console.warn(`警告: ${currentUsername} 验证失败。请检查cookie是否有效。`);
console.error(e);
await this.clearPerplexityCookies(page);
await browser.close();
}
} catch (e) {
console.error(`初始化浏览器失败 (${currentUsername})`);
console.error(e);
}
}
console.log(`验证完毕,有效cookie数量 ${Object.keys(this.sessions).filter((username) => this.sessions[username].valid).length}`);
// 开始网络监控
await this.networkMonitor.startMonitoring();
}
async clearPerplexityCookies(page) {
const client = await page.target().createCDPSession();
await client.send('Network.clearBrowserCookies');
await client.send('Network.clearBrowserCache');
const cookies = await page.cookies('https://www.perplexity.ai');
for (const cookie of cookies) {
await page.deleteCookie(cookie);
}
console.log('已自动清理 cookie');
}
async checkAccountStatus(page) {
if (process.env.USE_MANUAL_LOGIN !== "true" && process.env.INCOGNITO_MODE === "true") {
try {
await page.waitForSelector('div[class*="inline-flex w-full cursor-pointer select-none items-center justify-start gap-sm rounded-full"]', { timeout: 2000 });
// 检查无痕模式图标
const isIncognito = await page.evaluate(() => {
const incognitoElement = document.querySelector('div[class*="inline-flex w-full cursor-pointer select-none items-center justify-start gap-sm rounded-full"]');
return incognitoElement && incognitoElement.innerHTML.includes('viewBox="0 0 24 24"');
});
if (isIncognito) {
return { isPro: false, username: 'Incognito User', isIncognito: true };
} else {
return { isPro: false, username: null, isIncognito: false };
}
} catch (error) {
return { isPro: false, username: null, isIncognito: false };
}
} else {
try {
await page.waitForSelector('div[class*="relative flex items-center gap-x-xs"]', { timeout: 2000 });
const isPro = await page.evaluate(() => {
const svgElement = document.querySelector('div[class*="relative flex aspect-square"] svg');
return svgElement && svgElement.innerHTML.includes('fill-super dark:fill-superDark');
});
const username = await page.evaluate(() => {
const usernameElement = document.querySelector('div[class*="relative flex items-center gap-x-xs"] div');
return usernameElement ? usernameElement.textContent.trim() : null;
});
return { isPro, username, isIncognito: false };
} catch (error) {
return { isPro: false, username: null, isIncognito: false };
}
}
}
async waitForManualLogin(page) {
return new Promise(async (resolve) => {
let isResolved = false;
const checkLoginCompletion = async () => {
if (isResolved) return;
try {
const accountStatus = await this.checkAccountStatus(page);
if (accountStatus.username) {
console.log(`登录成功: ${accountStatus.username}`);
const cookies = await page.cookies();
const sessionCookie = this.extractPerplexitySessionCookie(cookies);
isResolved = true;
resolve({ sessionCookie, accountStatus });
}
} catch (error) {
// 如果检查失败,继续等待
}
};
// 监听 DOM 变化
await page.evaluate(() => {
const observer = new MutationObserver((mutations) => {
for (let mutation of mutations) {
if (mutation.type === 'childList' || mutation.type === 'subtree') {
window.dispatchEvent(new CustomEvent('domChanged'));
break;
}
}
});
observer.observe(document.body, {
childList: true,
subtree: true,
});
});
// 监听 DOM 变化
await page.exposeFunction('notifyDomChanged', checkLoginCompletion);
await page.evaluate(() => {
window.addEventListener('domChanged', window.notifyDomChanged);
});
// 监听导航完成
page.on('load', checkLoginCompletion);
setTimeout(() => {
if (!isResolved) {
console.log('登录等待超时');
resolve({ sessionCookie: null, accountStatus: null });
}
}, 300000);
// 初始检查
await checkLoginCompletion();
});
}
async validateSession(page) {
const accountStatus = await this.checkAccountStatus(page);
if (accountStatus.username || accountStatus.isIncognito) {
if (accountStatus.isIncognito) {
console.log('无痕模式验证成功');
} else {
console.log(`账号 ${accountStatus.username} 登录有效${accountStatus.isPro ? 'Pro 账号)' : ''}`);
}
return true;
} else {
console.log('账号验证失败');
return false;
}
}
extractPerplexitySessionCookie(cookies) {
let sessionCookie = [];
const sessionToken = cookies.find(c => c.name === '__Secure-next-auth.session-token');
if (sessionToken) {
sessionCookie.push(sessionToken);
}
if (sessionCookie.length === 0) {
console.error('无法提取有效的会话 cookie');
return null;
}
return sessionCookie;
}
async getCompletion({ username, messages, stream = false, proxyModel }) {
if (this.networkMonitor.isNetworkBlocked()) {
throw new Error("网络异常,请稍后再试");
}
const session = this.sessions[username];
if (!session || !session.valid) {
throw new Error(`用户 ${username} 的会话无效`);
}
const { page } = session;
const emitter = new EventEmitter();
// 转换纯文本
let previousMessages = messages.map(msg => msg.content).join("\n\n");
// 生成必要的参数
const messagePayload = {
version: "2.13",
source: "default",
language: "en-GB",
timezone: "Europe/London",
search_focus: "writing",
frontend_uuid: uuidV4(),
mode: "concise",
is_related_query: false,
is_default_related_query: false,
visitor_id: uuidV4(),
user_nextauth_id: uuidV4(),
frontend_context_uuid: uuidV4(),
prompt_source: "user",
query_source: "home",
};
await page.addScriptTag({ url: 'https://cdn.socket.io/4.4.1/socket.io.min.js' });
const ioExists = await page.evaluate(() => typeof io !== 'undefined');
if (!ioExists) {
throw new Error("socket.io 客户端脚本加载失败");
}
// 移除之前 console
page.removeAllListeners('console');
page.on('console', msg => {
const text = msg.text();
console.log(text);
});
// 唯一的回调函数名
const callbackName = `nodeCallback_${uuidV4()}`;
// 在浏览器上下文中建立 WebSocket 连接并发送消息
await page.exposeFunction(callbackName, (event, data) => {
if (event === "completion") {
const { id, text } = data;
emitter.emit("completion", id, text);
} else if (event === "end") {
emitter.emit("end");
} else if (event === "error") {
console.error("Error from page.evaluate:", data);
emitter.emit("error", data);
}
});
await page.evaluate(
(previousMessages, messagePayload, callbackName, stream) => {
try {
if (typeof io === 'undefined') {
window[callbackName]("error", "Socket.io 客户端未定义,请检查脚本是否正确加载。");
return;
}
if (window.socket) {
// 如果已有 socket 连接
window.socket.disconnect();
window.socket = null;
}
// 累积接收到的 chunk(用于非流)
let accumulatedChunks = '';
const socket = io("wss://www.perplexity.ai/", {
path: "/socket.io",
transports: ["websocket"],
});
window.socket = socket;
// 定义监听器函数
function onQueryProgress(data) {
try {
if (data.text) {
const textData = JSON.parse(data.text);
const chunk = textData.chunks[textData.chunks.length - 1];
if (chunk) {
if (stream) {
// 实时发送 chunk
console.log(chunk); // 直接输出 chunk 内容
window[callbackName]("completion", { id: messagePayload.frontend_uuid, text: chunk });
} else {
// 累积 chunk
accumulatedChunks += chunk;
}
}
}
// 检查是否为最后一条消息
if (data.final) {
if (!stream) {
// 非流发送完整的响应
console.log(accumulatedChunks);
window[callbackName]("completion", { id: messagePayload.frontend_uuid, text: accumulatedChunks });
}
console.log("请求结束");
window[callbackName]("end");
// 清理资源
socket.off("query_progress", onQueryProgress);
socket.disconnect();
window.socket = null;
delete window[callbackName];
}
} catch (err) {
console.error("Error processing query_progress data:", err);
window[callbackName]("error", err.toString());
}
}
// 添加监听器
socket.on("query_progress", onQueryProgress);
socket.on("connect", () => {
// 使用回调处理确认响应
socket.emit("perplexity_ask", previousMessages, messagePayload, (response) => {
});
});
socket.on("error", (error) => {
console.error("Socket error:", error);
window[callbackName]("error", error.toString());
});
socket.on("connect_error", (error) => {
console.error("Socket connect_error:", error);
window[callbackName]("error", error.toString());
});
socket.on("connect_timeout", (timeout) => {
console.error("Socket connect_timeout:", timeout);
window[callbackName]("error", "Connection timed out");
});
} catch (err) {
console.error("Error in page.evaluate:", err);
window[callbackName]("error", err.toString());
}
},
previousMessages,
messagePayload,
callbackName,
stream
);
const cancel = () => {
page.evaluate((callbackName) => {
if (window.socket) {
window.socket.disconnect();
window.socket = null;
}
if (window[callbackName]) {
delete window[callbackName];
}
}, callbackName).catch(console.error);
};
// 触发 'start'
emitter.emit("start", messagePayload.frontend_uuid);
return { completion: emitter, cancel };
}
}
export default PerplexityProvider;
-51
View File
@@ -1,51 +0,0 @@
import YouProvider from './you_providers/youProvider.mjs';
import PerplexityProvider from './perplexity_providers/perplexityProvider.mjs';
import HappyApiProvider from './happyapi_providers/happyApi.mjs';
import {config as youConfig} from './config.mjs';
import {config as perplexityConfig} from './perplexityConfig.mjs';
class ProviderManager {
constructor() {
// 根据环境变量初始化提供者
const activeProvider = process.env.ACTIVE_PROVIDER || 'you';
switch (activeProvider) {
case 'you':
this.provider = new YouProvider(youConfig);
break;
case 'perplexity':
this.provider = new PerplexityProvider(perplexityConfig);
break;
case 'happyapi':
this.provider = new HappyApiProvider();
break;
default:
throw new Error('Invalid ACTIVE_PROVIDER. Use "you", "perplexity", or "happyapi".');
}
console.log(`Initialized with ${activeProvider} provider.`);
}
async init() {
await this.provider.init(this.provider.config);
console.log(`Provider initialized.`);
}
async getCompletion(params) {
return this.provider.getCompletion(params);
}
getCurrentProvider() {
return this.provider.constructor.name;
}
getLogger() {
return this.provider.logger;
}
getSessionManager() {
return this.provider.sessionManager;
}
}
export default ProviderManager;
+429 -428
View File
@@ -1,428 +1,429 @@
export function formatMessages(messages, proxyModel, randomFileName) {
// 检查是否是 Claude 模型
const isClaudeModel = proxyModel.toLowerCase().includes('claude');
// 启用特殊前缀
const USE_BACKSPACE_PREFIX = process.env.USE_BACKSPACE_PREFIX === 'true';
// 定义角色映射
const roleFeatures = getRoleFeatures(messages, isClaudeModel, USE_BACKSPACE_PREFIX);
// 清除第一条消息的 role
messages = clearFirstMessageRole(messages);
messages = removeCustomRoleDefinitions(messages);
messages = convertRoles(messages, roleFeatures);
// 替换 content 中的角色
messages = replaceRolesInContent(messages, roleFeatures);
// 如果启用 clewd
const CLEWD_ENABLED = process.env.CLEWD_ENABLED === 'true';
if (CLEWD_ENABLED) {
messages = xmlPlotAllMessages(messages, roleFeatures);
}
messages = messages.map((message) => {
let newMessage = { ...message };
if (typeof newMessage.content === 'string') {
// 1) 移除 </FORMAT LINE BREAK/> 标识
let tempContent = newMessage.content.replace(/<\/FORMAT\s+LINE\s+BREAK\/>/g, '');
tempContent = tempContent.replace(/\n{3,}/g, '\n\n');
newMessage.content = tempContent;
}
return newMessage;
});
return messages;
}
/**
* 将首条消息role置空
* @param {Array} messages - 消息数组
* @returns {Array} 处理后的消息数组
*/
function clearFirstMessageRole(messages) {
if (!Array.isArray(messages) || messages.length === 0) {
return messages;
}
const processedMessages = messages.map(msg => ({...msg}));
processedMessages[0] = {
...processedMessages[0],
role: ''
};
return processedMessages;
}
// 获取角色特征
function getRoleFeatures(messages, isClaudeModel, useBackspacePrefix) {
let prefix = useBackspacePrefix ? '\u0008' : '';
let systemRole = `${prefix}${isClaudeModel ? 'System' : 'system'}`;
let userRole = `${prefix}${isClaudeModel ? 'Human' : 'user'}`;
let assistantRole = `${prefix}${isClaudeModel ? 'Assistant' : 'assistant'}`;
// 匹配自定义角色
const rolePattern = /\[\|(\w+)::(.*?)\|\]/g;
let customRoles = {};
messages.forEach(message => {
let content = message.content;
let match;
while ((match = rolePattern.exec(content)) !== null) {
const roleKey = match[1]; // 'system', 'user', 'assistant'
customRoles[roleKey.toLowerCase()] = match[2];
}
});
if (Object.keys(customRoles).length > 0) {
prefix = '';
if (customRoles['system']) {
systemRole = customRoles['system'];
}
if (customRoles['user']) {
userRole = customRoles['user'];
}
if (customRoles['assistant']) {
assistantRole = customRoles['assistant'];
}
}
return {
systemRole,
userRole,
assistantRole,
prefix
};
}
// 移除 messages 自定义角色格式
function removeCustomRoleDefinitions(messages) {
const rolePattern = /\[\|\w+::.*?\|]/g;
return messages.map(message => {
let newContent = message.content.replace(rolePattern, '');
return {
...message,
content: newContent
};
});
}
// 转换角色
function convertRoles(messages, roleFeatures) {
const {systemRole, userRole, assistantRole} = roleFeatures;
const roleMap = {
'system': systemRole,
'user': userRole,
'human': userRole,
'assistant': assistantRole
};
return messages.map(message => {
let currentRole = message.role;
if (currentRole.startsWith('\u0008')) {
// 包含前缀不需要转换
return message;
} else {
const roleKey = currentRole.toLowerCase();
const newRole = roleMap[roleKey] || currentRole;
return {...message, role: newRole};
}
});
}
// 替换 content 中的角色定义
function replaceRolesInContent(messages, roleFeatures) {
// 避免重复添加
const roleMap = {
'System:': roleFeatures.systemRole.replace(roleFeatures.prefix, '') + ':',
'system:': roleFeatures.systemRole.replace(roleFeatures.prefix, '') + ':',
'Human:': roleFeatures.userRole.replace(roleFeatures.prefix, '') + ':',
'human:': roleFeatures.userRole.replace(roleFeatures.prefix, '') + ':',
'user:': roleFeatures.userRole.replace(roleFeatures.prefix, '') + ':',
'Assistant:': roleFeatures.assistantRole.replace(roleFeatures.prefix, '') + ':',
'assistant:': roleFeatures.assistantRole.replace(roleFeatures.prefix, '') + ':',
};
// 构建角色正则
const escapedLabels = Object.keys(roleMap).map(label => escapeRegExp(label));
const prefixPattern = roleFeatures.prefix ? escapeRegExp(roleFeatures.prefix) : '';
const roleNamesPattern = new RegExp(`(\\n\\n)(${prefixPattern})?(${escapedLabels.join('|')})`, 'g');
return messages.map(message => {
let newContent = message.content;
if (typeof newContent === 'string') {
// 仅替换段落开头角色
newContent = newContent.replace(roleNamesPattern, (match, p1, p2, p3) => {
const newRoleLabel = roleMap[p3] || p3;
const prefixToUse = p2 !== undefined ? p2 : roleFeatures.prefix;
return p1 + (prefixToUse || '') + newRoleLabel;
});
} else {
console.warn('message.content is not a string:', newContent);
newContent = '';
}
return {
...message,
content: newContent
};
});
}
/**
* CLEWD 阶段对一组 messages 进行二次处理
* 若消息 content 不含 <|KEEP_ROLE|>则将 role 置空
* 然后调用 xmlHyperProcess 做多轮正则合并<@N>插入收尾清理
*
* @param {Array} messages - [{ role: 'user', content: '...' }, ...]
* @param {object} roleFeatures - { systemRole, userRole, assistantRole, prefix }
* @return {Array} 新的 messages
*/
export function xmlPlotAllMessages(messages, roleFeatures) {
return messages.map((msg) => {
if (!msg.content.includes('<|KEEP_ROLE|>')) {
msg = {
...msg,
role: '' // 置空
};
}
// 核心调用
const processed = xmlHyperProcess(msg.content, roleFeatures);
return { ...msg, content: processed };
});
}
/**
* 多轮 <regex order=1 / 2 / 3>
* MergeDisable 判断 + System => user 替换
* 段落合并 + <@N>插入
* 最终收尾 cleanup
*
* @param {string} originalContent
* @param {object} roleFeatures
* @returns {string} 处理后的文本
*/
function xmlHyperProcess(originalContent, roleFeatures) {
let content = originalContent;
let regexLogs = '';
// 第1轮正则 (order=1)
[content, regexLogs] = xmlHyperRegex(content, 1, regexLogs);
// 检测 MergeDisable 标记
const mergeDisable = {
all: content.includes('<|Merge Disable|>'),
system: content.includes('<|Merge System Disable|>'),
user: content.includes('<|Merge Human Disable|>'),
assistant: content.includes('<|Merge Assistant Disable|>')
};
// 把“system:”在部分情况下替换为“user:”
content = preSystemToUserFallback(content, roleFeatures, mergeDisable);
// 开始合并 (首次)
content = xmlHyperMerge(content, roleFeatures, mergeDisable);
// 处理 <@N> 插入
content = handleSubInsertion(content);
// 第2轮正则 (order=2)
[content, regexLogs] = xmlHyperRegex(content, 2, regexLogs);
// 第2次合并
content = xmlHyperMerge(content, roleFeatures, mergeDisable);
// 第3轮正则 (order=3)
[content, regexLogs] = xmlHyperRegex(content, 3, regexLogs);
// 插入对 <|padtxtX|> 的处理 or countTokens, etc.
// 收尾清理
content = finalizeCleanup(content);
return content.trim();
}
/**
* xmlHyperRegex:
* 匹配 <regex order=?> " /pattern/flags ":" replacement" </regex>
*/
function xmlHyperRegex(original, order, logs) {
let content = original;
// 仅处理同 order 的 block
const patternRegex = new RegExp(
`<regex(?: +order *= *(${order}))?>\\s*"\\/([^"]*?)\\/([gimsyu]*)"\\s*:\\s*"(.*?)"\\s*<\\/regex>`,
'gm'
);
let match;
while ((match = patternRegex.exec(content)) !== null) {
const entire = match[0];
const rawPattern = match[2];
const rawFlags = match[3];
let replacement = match[4];
logs += `${entire}\n`;
try {
const regObj = new RegExp(rawPattern, rawFlags);
replacement = JSON.parse(`"${replacement.replace(/\\?"/g, '\\"')}"`); // 反序列化
content = content.replace(regObj, replacement);
} catch (err) {
console.warn(`Regex parse/replace error in block: ${entire}\n`, err);
}
}
return [content, logs];
}
/**
* content = content.replace(...)
* 而后 system: => user:
* 根据 roleFeatures.systemRole / userRole 动态替换
*/
function preSystemToUserFallback(content, roleFeatures, mergeDisable) {
if (mergeDisable.all || mergeDisable.system || mergeDisable.user) {
return content; // 不做任何替换
}
const systemName = stripPrefix(roleFeatures.systemRole, roleFeatures.prefix);
const userName = stripPrefix(roleFeatures.userRole, roleFeatures.prefix);
const assistantName = stripPrefix(roleFeatures.assistantRole, roleFeatures.prefix);
// 前面非 user|assistant 段落时的 system: -> 去掉
const re1 = new RegExp(
`(\\n\\n|^\\s*)(?<!\\n\\n(${userName}|${assistantName}):.*?)${systemName}:\\s*`,
'gs'
);
content = content.replace(re1, '$1');
// 补充 system => user
const re2 = new RegExp(`(\\n\\n|^\\s*)${systemName}:\\s*`, 'g');
content = content.replace(re2, `\n\n${userName}: `);
return content;
}
/**
* xmlHyperMerge:
* 利用段落合并逻辑动态 roleFeatures
*/
function xmlHyperMerge(original, roleFeatures, mergeDisable) {
let content = original;
if (mergeDisable.all) {
return content; // 禁用合并
}
// 先获取名称
const sys = stripPrefix(roleFeatures.systemRole, roleFeatures.prefix);
const usr = stripPrefix(roleFeatures.userRole, roleFeatures.prefix);
const ast = stripPrefix(roleFeatures.assistantRole, roleFeatures.prefix);
// 合并 system 段
if (!mergeDisable.system) {
const regSys = new RegExp(`(?:\\n\\n|^\\s*)${escapeRegExp(sys)}:\\s*(.*?)(?=\\n\\n(?:${escapeRegExp(usr)}|${escapeRegExp(ast)}|$))`, 'gs');
content = content.replace(regSys, (_m, p1) => `\n\n${sys}: ${p1}`);
}
// 合并 user 段
if (!mergeDisable.user) {
const regUsr = new RegExp(`(?:\\n\\n|^\\s*)${escapeRegExp(usr)}:\\s*(.*?)(?=\\n\\n(?:${escapeRegExp(ast)}|${escapeRegExp(sys)}|$))`, 'gs');
content = content.replace(regUsr, (_m, p1) => `\n\n${usr}: ${p1}`);
}
// 合并 assistant 段
if (!mergeDisable.assistant) {
const regAst = new RegExp(`(?:\\n\\n|^\\s*)${escapeRegExp(ast)}:\\s*(.*?)(?=\\n\\n(?:${escapeRegExp(usr)}|${escapeRegExp(sys)}|$))`, 'gs');
content = content.replace(regAst, (_m, p1) => `\n\n${ast}: ${p1}`);
}
return content;
}
/**
* splitContent = content.split(regExp)
* 然后 for each <@N> => 把里面内容插入到 倒数第N段落后面
* 删除 match[0]
*/
function handleSubInsertion(original) {
let content = original;
// 按 \n\n(?=.*?:) 拆分
let splitted = content.split(/\n\n(?=.*?:)/g);
let match;
while ((match = /<@(\d+)>(.*?)<\/@\1>/gs.exec(content)) !== null) {
const idx = splitted.length - parseInt(match[1], 10) - 1;
if (idx >= 0 && splitted[idx]) {
splitted[idx] += `\n\n${match[2]}`;
}
content = content.replace(match[0], '');
}
// 重组
content = splitted.join('\n\n').replace(/<@(\d+)>.*?<\/@\1>/gs, '');
return content;
}
/**
* 移除 <regex> 统一换行<|curtail|> => 换行<|join|> =>
* <|space|> => ' '以及 <|xxx|> => JSON.parse
* 然后去除多余空行
*/
function finalizeCleanup(original) {
let content = original;
// 移除 <regex> 块
content = content.replace(/<regex( +order *= *\d)?>.*?<\/regex>/gm, '');
// 统一换行
content = content.replace(/\r\n|\r/gm, '\n');
// <|curtail|> => 换行
content = content.replace(/\s*<\|curtail\|>\s*/g, '\n');
// <|join|> => ''
content = content.replace(/\s*<\|join\|>\s*/g, '');
// <|space|> => ' '
content = content.replace(/\s*<\|space\|>\s*/g, ' ');
// JSON反序列化
content = content.replace(/<\|(\\.*?)\|>/g, (m, p1) => {
try {
return JSON.parse(`"${p1.replace(/\\?"/g, '\\"')}"`);
} catch {
return m; // 保留原文本
}
});
// 移除其他 <|xxx|> 标记
content = content.replace(/\s*<\|(?!padtxt).*?\|>\s*/g, '\n\n');
// 去除多余空行
content = content.trim().replace(/(?<=\n)\n(?=\n)/g, '');
return content;
}
/** stripPrefix: 如果 role 中含有前缀(如 \u0008),去掉 */
function stripPrefix(fullStr, prefix) {
if (!prefix) return fullStr;
if (fullStr.startsWith(prefix)) {
return fullStr.slice(prefix.length);
}
return fullStr;
}
/** 转义正则特殊字符 */
function escapeRegExp(str) {
return str.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
import sysLogger from './utils/sysLogger.mjs';
export function formatMessages(messages, proxyModel, randomFileName) {
// Check if it is a Claude model
const isClaudeModel = proxyModel.toLowerCase().includes('claude');
// Enable special prefix
const USE_BACKSPACE_PREFIX = process.env.USE_BACKSPACE_PREFIX === 'true';
// Define role mapping
const roleFeatures = getRoleFeatures(messages, isClaudeModel, USE_BACKSPACE_PREFIX);
// Clear role of first message
messages = clearFirstMessageRole(messages);
messages = removeCustomRoleDefinitions(messages);
messages = convertRoles(messages, roleFeatures);
// Replace role in content
messages = replaceRolesInContent(messages, roleFeatures);
// If clewd is enabled
const CLEWD_ENABLED = process.env.CLEWD_ENABLED === 'true';
if (CLEWD_ENABLED) {
messages = xmlPlotAllMessages(messages, roleFeatures);
}
messages = messages.map((message) => {
let newMessage = { ...message };
if (typeof newMessage.content === 'string') {
// 1) Remove </FORMAT LINE BREAK/> tag
let tempContent = newMessage.content.replace(/<\/FORMAT\s+LINE\s+BREAK\/>/g, '');
tempContent = tempContent.replace(/\n{3,}/g, '\n\n');
newMessage.content = tempContent;
}
return newMessage;
});
return messages;
}
/**
* roleSet to empty
* @param {Array} messages -
* @returns {Array}
*/
function clearFirstMessageRole(messages) {
if (!Array.isArray(messages) || messages.length === 0) {
return messages;
}
const processedMessages = messages.map(msg => ({...msg}));
processedMessages[0] = {
...processedMessages[0],
role: ''
};
return processedMessages;
}
// Get role characteristics
function getRoleFeatures(messages, isClaudeModel, useBackspacePrefix) {
let prefix = useBackspacePrefix ? '\u0008' : '';
let systemRole = `${prefix}${isClaudeModel ? 'System' : 'system'}`;
let userRole = `${prefix}${isClaudeModel ? 'Human' : 'user'}`;
let assistantRole = `${prefix}${isClaudeModel ? 'Assistant' : 'assistant'}`;
// Match custom role
const rolePattern = /\[\|(\w+)::(.*?)\|\]/g;
let customRoles = {};
messages.forEach(message => {
let content = message.content;
let match;
while ((match = rolePattern.exec(content)) !== null) {
const roleKey = match[1]; // 'system', 'user', 'assistant'
customRoles[roleKey.toLowerCase()] = match[2];
}
});
if (Object.keys(customRoles).length > 0) {
prefix = '';
if (customRoles['system']) {
systemRole = customRoles['system'];
}
if (customRoles['user']) {
userRole = customRoles['user'];
}
if (customRoles['assistant']) {
assistantRole = customRoles['assistant'];
}
}
return {
systemRole,
userRole,
assistantRole,
prefix
};
}
// Remove custom role format from messages
function removeCustomRoleDefinitions(messages) {
const rolePattern = /\[\|\w+::.*?\|]/g;
return messages.map(message => {
let newContent = message.content.replace(rolePattern, '');
return {
...message,
content: newContent
};
});
}
// Convert role
function convertRoles(messages, roleFeatures) {
const {systemRole, userRole, assistantRole} = roleFeatures;
const roleMap = {
'system': systemRole,
'user': userRole,
'human': userRole,
'assistant': assistantRole
};
return messages.map(message => {
let currentRole = message.role;
if (currentRole.startsWith('\u0008')) {
// Contains prefix, no conversion needed
return message;
} else {
const roleKey = currentRole.toLowerCase();
const newRole = roleMap[roleKey] || currentRole;
return {...message, role: newRole};
}
});
}
// Replace role definitions in content
function replaceRolesInContent(messages, roleFeatures) {
// Avoid duplicate addition
const roleMap = {
'System:': roleFeatures.systemRole.replace(roleFeatures.prefix, '') + ':',
'system:': roleFeatures.systemRole.replace(roleFeatures.prefix, '') + ':',
'Human:': roleFeatures.userRole.replace(roleFeatures.prefix, '') + ':',
'human:': roleFeatures.userRole.replace(roleFeatures.prefix, '') + ':',
'user:': roleFeatures.userRole.replace(roleFeatures.prefix, '') + ':',
'Assistant:': roleFeatures.assistantRole.replace(roleFeatures.prefix, '') + ':',
'assistant:': roleFeatures.assistantRole.replace(roleFeatures.prefix, '') + ':',
};
// Build role regex
const escapedLabels = Object.keys(roleMap).map(label => escapeRegExp(label));
const prefixPattern = roleFeatures.prefix ? escapeRegExp(roleFeatures.prefix) : '';
const roleNamesPattern = new RegExp(`(\\n\\n)(${prefixPattern})?(${escapedLabels.join('|')})`, 'g');
return messages.map(message => {
let newContent = message.content;
if (typeof newContent === 'string') {
// Only replace role at beginning of paragraph
newContent = newContent.replace(roleNamesPattern, (match, p1, p2, p3) => {
const newRoleLabel = roleMap[p3] || p3;
const prefixToUse = p2 !== undefined ? p2 : roleFeatures.prefix;
return p1 + (prefixToUse || '') + newRoleLabel;
});
} else {
sysLogger.warn('message.content is not a string:', newContent);
newContent = '';
}
return {
...message,
content: newContent
};
});
}
/**
* CLEWD messages
* content <|KEEP_ROLE|> role Set to empty
* xmlHyperProcess <@N>Final cleanup
*
* @param {Array} messages - [{ role: 'user', content: '...' }, ...]
* @param {object} roleFeatures - { systemRole, userRole, assistantRole, prefix }
* @return {Array} messages
*/
export function xmlPlotAllMessages(messages, roleFeatures) {
return messages.map((msg) => {
if (!msg.content.includes('<|KEEP_ROLE|>')) {
msg = {
...msg,
role: '' // Set to empty
};
}
// Core call
const processed = xmlHyperProcess(msg.content, roleFeatures);
return { ...msg, content: processed };
});
}
/**
* <regex order=1 / 2 / 3>
* MergeDisable + System => user
* + <@N>
* cleanup
*
* @param {string} originalContent
* @param {object} roleFeatures
* @returns {string}
*/
function xmlHyperProcess(originalContent, roleFeatures) {
let content = originalContent;
let regexLogs = '';
// 1st round regex (order=1)
[content, regexLogs] = xmlHyperRegex(content, 1, regexLogs);
// Detect MergeDisable tag
const mergeDisable = {
all: content.includes('<|Merge Disable|>'),
system: content.includes('<|Merge System Disable|>'),
user: content.includes('<|Merge Human Disable|>'),
assistant: content.includes('<|Merge Assistant Disable|>')
};
// Replace 'system:' with 'user:' in some cases
content = preSystemToUserFallback(content, roleFeatures, mergeDisable);
// Start merging (first time)
content = xmlHyperMerge(content, roleFeatures, mergeDisable);
// Process <@N> insertion
content = handleSubInsertion(content);
// 2nd round regex (order=2)
[content, regexLogs] = xmlHyperRegex(content, 2, regexLogs);
// 2nd merge
content = xmlHyperMerge(content, roleFeatures, mergeDisable);
// 3rd round regex (order=3)
[content, regexLogs] = xmlHyperRegex(content, 3, regexLogs);
// Insert handling for <|padtxtX|> or countTokens, etc.
// Final cleanup
content = finalizeCleanup(content);
return content.trim();
}
/**
* xmlHyperRegex:
* <regex order=?> " /pattern/flags ":" replacement" </regex>
*/
function xmlHyperRegex(original, order, logs) {
let content = original;
// Only process blocks with same order
const patternRegex = new RegExp(
`<regex(?: +order *= *(${order}))?>\\s*"\\/([^"]*?)\\/([gimsyu]*)"\\s*:\\s*"(.*?)"\\s*<\\/regex>`,
'gm'
);
let match;
while ((match = patternRegex.exec(content)) !== null) {
const entire = match[0];
const rawPattern = match[2];
const rawFlags = match[3];
let replacement = match[4];
logs += `${entire}\n`;
try {
const regObj = new RegExp(rawPattern, rawFlags);
replacement = JSON.parse(`"${replacement.replace(/\\?"/g, '\\"')}"`); // Deserialize
content = content.replace(regObj, replacement);
} catch (err) {
sysLogger.warn(`Regex parse/replace error in block: ${entire}\n`, err);
}
}
return [content, logs];
}
/**
* content = content.replace(...)
* system: => user:
* roleFeatures.systemRole / userRole
*/
function preSystemToUserFallback(content, roleFeatures, mergeDisable) {
if (mergeDisable.all || mergeDisable.system || mergeDisable.user) {
return content; // Do not replace
}
const systemName = stripPrefix(roleFeatures.systemRole, roleFeatures.prefix);
const userName = stripPrefix(roleFeatures.userRole, roleFeatures.prefix);
const assistantName = stripPrefix(roleFeatures.assistantRole, roleFeatures.prefix);
// Remove system: prefix when preceding non-user|assistant paragraphs
const re1 = new RegExp(
`(\\n\\n|^\\s*)(?<!\\n\\n(${userName}|${assistantName}):.*?)${systemName}:\\s*`,
'gs'
);
content = content.replace(re1, '$1');
// Supplement system => user
const re2 = new RegExp(`(\\n\\n|^\\s*)${systemName}:\\s*`, 'g');
content = content.replace(re2, `\n\n${userName}: `);
return content;
}
/**
* xmlHyperMerge:
* roleFeatures
*/
function xmlHyperMerge(original, roleFeatures, mergeDisable) {
let content = original;
if (mergeDisable.all) {
return content; // Disable merge
}
// First get name
const sys = stripPrefix(roleFeatures.systemRole, roleFeatures.prefix);
const usr = stripPrefix(roleFeatures.userRole, roleFeatures.prefix);
const ast = stripPrefix(roleFeatures.assistantRole, roleFeatures.prefix);
// Merge system section
if (!mergeDisable.system) {
const regSys = new RegExp(`(?:\\n\\n|^\\s*)${escapeRegExp(sys)}:\\s*(.*?)(?=\\n\\n(?:${escapeRegExp(usr)}|${escapeRegExp(ast)}|$))`, 'gs');
content = content.replace(regSys, (_m, p1) => `\n\n${sys}: ${p1}`);
}
// Merge user section
if (!mergeDisable.user) {
const regUsr = new RegExp(`(?:\\n\\n|^\\s*)${escapeRegExp(usr)}:\\s*(.*?)(?=\\n\\n(?:${escapeRegExp(ast)}|${escapeRegExp(sys)}|$))`, 'gs');
content = content.replace(regUsr, (_m, p1) => `\n\n${usr}: ${p1}`);
}
// Merge assistant section
if (!mergeDisable.assistant) {
const regAst = new RegExp(`(?:\\n\\n|^\\s*)${escapeRegExp(ast)}:\\s*(.*?)(?=\\n\\n(?:${escapeRegExp(usr)}|${escapeRegExp(sys)}|$))`, 'gs');
content = content.replace(regAst, (_m, p1) => `\n\n${ast}: ${p1}`);
}
return content;
}
/**
* splitContent = content.split(regExp)
* for each <@N> => N
* match[0]
*/
function handleSubInsertion(original) {
let content = original;
// Split by \n\n(?=.*?:)
let splitted = content.split(/\n\n(?=.*?:)/g);
let match;
while ((match = /<@(\d+)>(.*?)<\/@\1>/gs.exec(content)) !== null) {
const idx = splitted.length - parseInt(match[1], 10) - 1;
if (idx >= 0 && splitted[idx]) {
splitted[idx] += `\n\n${match[2]}`;
}
content = content.replace(match[0], '');
}
// Reassemble
content = splitted.join('\n\n').replace(/<@(\d+)>.*?<\/@\1>/gs, '');
return content;
}
/**
* Remove <regex> blockUnify line breaks<|curtail|> => newline<|join|> =>
* <|space|> => ' ' <|xxx|> => JSON.parse
* Remove extra blank lines
*/
function finalizeCleanup(original) {
let content = original;
// Remove <regex> block
content = content.replace(/<regex( +order *= *\d)?>.*?<\/regex>/gm, '');
// Unify line breaks
content = content.replace(/\r\n|\r/gm, '\n');
// <|curtail|> => newline
content = content.replace(/\s*<\|curtail\|>\s*/g, '\n');
// <|join|> => ''
content = content.replace(/\s*<\|join\|>\s*/g, '');
// <|space|> => ' '
content = content.replace(/\s*<\|space\|>\s*/g, ' ');
// JSON deserialization
content = content.replace(/<\|(\\.*?)\|>/g, (m, p1) => {
try {
return JSON.parse(`"${p1.replace(/\\?"/g, '\\"')}"`);
} catch {
return m; // Keep original text
}
});
// Remove other <|xxx|> tags
content = content.replace(/\s*<\|(?!padtxt).*?\|>\s*/g, '\n\n');
// Remove extra blank lines
content = content.trim().replace(/(?<=\n)\n(?=\n)/g, '');
return content;
}
/** stripPrefix: role ( \u0008) */
function stripPrefix(fullStr, prefix) {
if (!prefix) return fullStr;
if (fullStr.startsWith(prefix)) {
return fullStr.slice(prefix.length);
}
return fullStr;
}
/** */
function escapeRegExp(str) {
return str.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
+67
View File
@@ -0,0 +1,67 @@
import sysLogger from './utils/sysLogger.mjs';
const imageStorage = new Map();
let imageCounter = 0;
/**
*
* @param {string} base64Data base64
* @param {string} mediaType ("image/jpeg")
* @returns {{ imageId: string, mediaType: string }}
*/
export function storeImage(base64Data, mediaType) {
const randomPart = Math.floor(Math.random() * 10000);
const imageId = `image_${Date.now()}_${imageCounter++}_${randomPart}`;
imageStorage.set(imageId, { imageId, base64Data, mediaType });
// sysLogger.debug(`Image stored with ID: ${imageId}, Media Type: ${mediaType}`);
// Print stored base64
// sysLogger.debug(`Base64 Data for Image ID ${imageId}: ${base64Data.substring(0, 100)}...`);
return { imageId, mediaType };
}
/**
*
* @param {string} imageId ID
* @returns {object|null} base64
*/
export function getImage(imageId) {
return imageStorage.get(imageId) || null;
}
/**
*
* @returns {object|null} base64
*/
export function getLastImage() {
const lastKey = Array.from(imageStorage.keys()).pop();
return lastKey ? imageStorage.get(lastKey) : null;
}
/**
*
* @param {string} imageId ID
*/
export function removeImage(imageId) {
imageStorage.delete(imageId);
}
/**
*
*/
export function clearAllImages() {
imageStorage.clear();
imageCounter = 0;
// sysLogger.debug("All images have been cleared from memory.");
}
/**
*
*/
export function printAllImages() {
sysLogger.debug('Current images stored in memory:');
imageStorage.forEach((value, key) => {
sysLogger.debug(`Image ID: ${key}, Media Type: ${value.mediaType}`);
sysLogger.debug(`Base64 Data: ${value.base64Data.substring(0, 100)}...`);
});
}
+1031 -926
View File
File diff suppressed because it is too large Load Diff
+51 -50
View File
@@ -1,51 +1,52 @@
import dns from 'dns';
import { EventEmitter } from 'events';
class NetworkMonitor extends EventEmitter {
constructor() {
super();
this.isBlocked = false;
this.checkInterval = null;
}
async checkConnection() {
return new Promise((resolve) => {
dns.lookup('you.com', (err) => {
if (err && err.code === 'ENOTFOUND') {
resolve(false);
} else {
resolve(true);
}
});
});
}
async startMonitoring() {
console.log("开始网络监控...");
this.checkInterval = setInterval(async () => {
const isConnected = await this.checkConnection();
if (!isConnected && !this.isBlocked) {
this.isBlocked = true;
this.emit('networkDown');
console.log("检测到网络异常");
} else if (isConnected && this.isBlocked) {
this.isBlocked = false;
this.emit('networkUp');
console.log("网络恢复正常");
}
}, 5000);
}
stopMonitoring() {
if (this.checkInterval) {
clearInterval(this.checkInterval);
this.checkInterval = null;
}
}
isNetworkBlocked() {
return this.isBlocked;
}
}
import dns from 'dns';
import { EventEmitter } from 'events';
import sysLogger from './utils/sysLogger.mjs';
class NetworkMonitor extends EventEmitter {
constructor() {
super();
this.isBlocked = false;
this.checkInterval = null;
}
async checkConnection() {
return new Promise((resolve) => {
dns.lookup('you.com', (err) => {
if (err && err.code === 'ENOTFOUND') {
resolve(false);
} else {
resolve(true);
}
});
});
}
async startMonitoring() {
sysLogger.debug("Starting network monitor...");
this.checkInterval = setInterval(async () => {
const isConnected = await this.checkConnection();
if (!isConnected && !this.isBlocked) {
this.isBlocked = true;
this.emit('networkDown');
sysLogger.debug("Network anomaly detected");
} else if (isConnected && this.isBlocked) {
this.isBlocked = false;
this.emit('networkUp');
sysLogger.debug("Network recovered");
}
}, 5000);
}
stopMonitoring() {
if (this.checkInterval) {
clearInterval(this.checkInterval);
this.checkInterval = null;
}
}
isNetworkBlocked() {
return this.isBlocked;
}
}
export default NetworkMonitor;
+112 -111
View File
@@ -1,111 +1,112 @@
import { SocksProxyAgent } from 'socks-proxy-agent';
import HttpsProxyAgent from 'https-proxy-agent';
import { URL } from 'url';
import http from 'http';
import https from 'https';
let globalProxyAgent = null;
function getProxyUrl() {
const proxyUrl = process.env.https_proxy || process.env.HTTPS_PROXY || process.env.http_proxy || process.env.HTTP_PROXY;
return proxyUrl ? proxyUrl.trim() : null;
}
function parseProxyUrl(proxyUrl) {
if (!proxyUrl) return null;
try {
let protocol, host, port, username, password;
if (proxyUrl.startsWith('socks5://')) {
const parts = proxyUrl.slice(9).split(':');
if (parts.length === 4) {
[host, port, username, password] = parts;
protocol = 'socks5:';
} else {
throw new Error('Invalid SOCKS5 proxy URL format');
}
} else {
const url = new URL(proxyUrl);
protocol = url.protocol;
host = url.hostname;
port = url.port;
username = url.username;
password = url.password;
}
return { protocol: protocol.replace(':', ''), host, port, username, password };
} catch (error) {
console.error(`Invalid proxy URL: ${proxyUrl}`);
return null;
}
}
function createProxyAgent() {
const proxyUrl = getProxyUrl();
if (!proxyUrl) {
console.log('Proxy environment variable not set, will not use proxy.');
return null;
}
const parsedProxy = parseProxyUrl(proxyUrl);
if (!parsedProxy) return null;
console.log(`Using proxy: ${proxyUrl}`);
if (parsedProxy.protocol === 'socks5') {
console.log('Using SOCKS5 proxy');
return new SocksProxyAgent({
hostname: parsedProxy.host,
port: parsedProxy.port,
userId: parsedProxy.username,
password: parsedProxy.password,
protocol: 'socks5:'
});
} else {
console.log('Using HTTP/HTTPS proxy');
return new HttpsProxyAgent.HttpsProxyAgent(proxyUrl);
}
}
export function setGlobalProxy() {
const proxyUrl = getProxyUrl();
if (proxyUrl) {
globalProxyAgent = createProxyAgent();
// 重写 http 和 https 模块的 request 方法
const originalHttpRequest = http.request;
const originalHttpsRequest = https.request;
http.request = function(options, callback) {
if (typeof options === 'string') {
options = new URL(options);
}
options.agent = globalProxyAgent;
return originalHttpRequest.call(this, options, callback);
};
https.request = function(options, callback) {
if (typeof options === 'string') {
options = new URL(options);
}
options.agent = globalProxyAgent;
return originalHttpsRequest.call(this, options, callback);
};
console.log(`Global proxy set to: ${proxyUrl}`);
} else {
console.log('No proxy environment variable set, global proxy not configured.');
}
}
export function setProxyEnvironmentVariables() {
const proxyUrl = getProxyUrl();
if (proxyUrl) {
process.env.HTTP_PROXY = proxyUrl;
process.env.HTTPS_PROXY = proxyUrl;
console.log(`Set proxy environment variables to: ${proxyUrl}`);
}
}
// 全局代理
setGlobalProxy();
import { SocksProxyAgent } from 'socks-proxy-agent';
import HttpsProxyAgent from 'https-proxy-agent';
import { URL } from 'url';
import http from 'http';
import https from 'https';
import sysLogger from './utils/sysLogger.mjs';
let globalProxyAgent = null;
function getProxyUrl() {
const proxyUrl = process.env.https_proxy || process.env.HTTPS_PROXY || process.env.http_proxy || process.env.HTTP_PROXY;
return proxyUrl ? proxyUrl.trim() : null;
}
function parseProxyUrl(proxyUrl) {
if (!proxyUrl) return null;
try {
let protocol, host, port, username, password;
if (proxyUrl.startsWith('socks5://')) {
const parts = proxyUrl.slice(9).split(':');
if (parts.length === 4) {
[host, port, username, password] = parts;
protocol = 'socks5:';
} else {
throw new Error('Invalid SOCKS5 proxy URL format');
}
} else {
const url = new URL(proxyUrl);
protocol = url.protocol;
host = url.hostname;
port = url.port;
username = url.username;
password = url.password;
}
return { protocol: protocol.replace(':', ''), host, port, username, password };
} catch (error) {
sysLogger.error(`Invalid proxy URL: ${proxyUrl}`);
return null;
}
}
function createProxyAgent() {
const proxyUrl = getProxyUrl();
if (!proxyUrl) {
sysLogger.info('Proxy environment variable not set, will not use proxy.');
return null;
}
const parsedProxy = parseProxyUrl(proxyUrl);
if (!parsedProxy) return null;
sysLogger.info(`Using proxy: ${proxyUrl}`);
if (parsedProxy.protocol === 'socks5') {
sysLogger.info('Using SOCKS5 proxy');
return new SocksProxyAgent({
hostname: parsedProxy.host,
port: parsedProxy.port,
userId: parsedProxy.username,
password: parsedProxy.password,
protocol: 'socks5:'
});
} else {
sysLogger.info('Using HTTP/HTTPS proxy');
return new HttpsProxyAgent.HttpsProxyAgent(proxyUrl);
}
}
export function setGlobalProxy() {
const proxyUrl = getProxyUrl();
if (proxyUrl) {
globalProxyAgent = createProxyAgent();
// Rewrite request method of http and https modules
const originalHttpRequest = http.request;
const originalHttpsRequest = https.request;
http.request = function(options, callback) {
if (typeof options === 'string') {
options = new URL(options);
}
options.agent = globalProxyAgent;
return originalHttpRequest.call(this, options, callback);
};
https.request = function(options, callback) {
if (typeof options === 'string') {
options = new URL(options);
}
options.agent = globalProxyAgent;
return originalHttpsRequest.call(this, options, callback);
};
sysLogger.info(`Global proxy set to: ${proxyUrl}`);
} else {
sysLogger.info('No proxy environment variable set, global proxy not configured.');
}
}
export function setProxyEnvironmentVariables() {
const proxyUrl = getProxyUrl();
if (proxyUrl) {
process.env.HTTP_PROXY = proxyUrl;
process.env.HTTPS_PROXY = proxyUrl;
sysLogger.info(`Set proxy environment variables to: ${proxyUrl}`);
}
}
// Global proxy
setGlobalProxy();
+84 -101
View File
@@ -1,86 +1,65 @@
import fs from 'fs';
import path from 'path';
import {Mutex} from 'async-mutex';
import winston from 'winston';
// 请求日志记录
const logFilePath = path.join(process.cwd(), 'request_logs.log');
/*
* 黑名单文件格式
* {
* "ip1": {
* "permanent": true
* },
* "ip2": {
* "permanent": true
* }
* ...
*/
const blacklistFilePath = path.join(process.cwd(), 'ip_blacklist.json');
/*
* 临时限制文件格式
* {
* "ip1": 1630000000000,
* "ip2": 1630000000000
* ...
* }
*/
const tempLimitFilePath = path.join(process.cwd(), 'temp_limits.json');
// 是否启用检测
import sysLogger from './utils/sysLogger.mjs';
// Logging
const logFilePath = path.join(process.cwd(), 'data', 'request_logs.log');
const blacklistFilePath = path.join(process.cwd(), 'data', 'ip_blacklist.json');
const tempLimitFilePath = path.join(process.cwd(), 'data', 'temp_limits.json');
const ENABLE_DETECTION = process.env.ENABLE_DETECTION !== 'false';
class RequestLogger {
constructor() {
// 日志记录
this.logger = winston.createLogger({
level: 'info',
format: winston.format.combine(
winston.format.timestamp({format: 'YYYY年MM月DD日 HH时mm分ss秒'}),
winston.format.printf(({timestamp, level, message}) => {
return `${timestamp} | ${message}`;
})
),
transports: [
new winston.transports.File({filename: logFilePath})
]
});
// Logging
this.logger = {
info: async (message) => {
const timestamp = new Date().toLocaleString('zh-CN', {
year: 'numeric', month: '2-digit', day: '2-digit',
hour: '2-digit', minute: '2-digit', second: '2-digit', hour12: false
}).replace(/\//g, '-');
const logLine = `${timestamp} | ${message}\n`;
try {
await fs.promises.appendFile(logFilePath, logLine);
} catch (e) {
sysLogger.error('Failed to write to request log:', e.message);
}
}
};
try {
if (!fs.existsSync(blacklistFilePath)) {
fs.writeFileSync(blacklistFilePath, JSON.stringify({}), 'utf8');
}
} catch (err) {
console.error(`初始化黑名单文件出错:${err.message}`);
sysLogger.error(`Error initializing blacklist file: ${err.message}`);
}
this.ipRequestRecords = {}; // 每个IP的请求时间记录
this.ip30sTriggerCount = {}; // 统计每个IP触发30秒限制的次数
this.temporaryLimits = {}; // 每个IP的临时限制到期时间
this.ipMutexes = {}; // 为每个IP分配一个独立的Mutex
this.blacklistMutex = new Mutex(); // 黑名单文件的Mutex
this.ipRequestRecords = {}; // Request time record per IP
this.ip30sTriggerCount = {}; // Count 30s limit triggers for each IP
this.temporaryLimits = {}; // Temp limit expiration time per IP
this.ipMutexes = {}; // Assign an independent Mutex for each IP
this.blacklistMutex = new Mutex(); // Blacklist file Mutex
try {
if (!fs.existsSync(tempLimitFilePath)) {
fs.writeFileSync(tempLimitFilePath, JSON.stringify({}), 'utf8');
}
} catch (err) {
console.error(`初始化临时限制文件出错:${err.message}`);
sysLogger.error(`Error initializing temp limit file: ${err.message}`);
}
// 加载临时限制
// Load temp limits
this.loadTempLimitsFromFile();
// 监听文件变化
// Listen to file changes
this.setupTempLimitsFileWatcher();
// 清理无活动IP
// Clean inactive IPs
setInterval(() => {
this.cleanUpInactiveIPs();
}, 60 * 60 * 1000);
}
// 获取IPMutex
// Get IP Mutex
getMutexForIP(ip) {
if (!this.ipMutexes[ip]) {
this.ipMutexes[ip] = new Mutex();
@@ -88,17 +67,17 @@ class RequestLogger {
return this.ipMutexes[ip];
}
// 从文件加载临时限制
loadTempLimitsFromFile() {
// Load temporary limits from file
async loadTempLimitsFromFile() {
try {
const dataRaw = fs.readFileSync(tempLimitFilePath, 'utf8');
const dataRaw = await fs.promises.readFile(tempLimitFilePath, 'utf8');
if (!dataRaw.trim()) {
// 文件为空,则 no-op
// File is empty, no-op
return;
}
const parsed = JSON.parse(dataRaw);
const now = Date.now();
// 清空内存
// Clear memory
this.temporaryLimits = {};
for (const ip in parsed) {
const expireTime = parseInt(parsed[ip], 10);
@@ -106,28 +85,28 @@ class RequestLogger {
this.temporaryLimits[ip] = expireTime;
}
}
this.saveTempLimitsToFile();
await this.saveTempLimitsToFile();
} catch (error) {
console.error(`读取临时限制文件出错:${error.message}`);
sysLogger.error(`Error reading temp limit file: ${error.message}`);
}
}
saveTempLimitsToFile() {
async saveTempLimitsToFile() {
try {
const now = Date.now();
// 移除过期记录
// Remove expired records
for (const ip in this.temporaryLimits) {
if (this.temporaryLimits[ip] <= now) {
delete this.temporaryLimits[ip];
}
}
fs.writeFileSync(
await fs.promises.writeFile(
tempLimitFilePath,
JSON.stringify(this.temporaryLimits, null, 4),
'utf8'
);
} catch (error) {
console.error(`写入临时限制文件出错:${error.message}`);
sysLogger.error(`Error writing temp limit file: ${error.message}`);
}
}
@@ -139,61 +118,65 @@ class RequestLogger {
}
});
} catch (err) {
console.error(`监听临时限制文件出错:${err.message}`);
sysLogger.error(`Error listening to temp limit file: ${err.message}`);
}
}
// 检查是否在黑名单
// Check if in blacklist
async isBlacklisted(ip) {
return await this.blacklistMutex.runExclusive(async () => {
try {
if (!fs.existsSync(blacklistFilePath)) {
fs.writeFileSync(blacklistFilePath, JSON.stringify({}), 'utf8');
try {
await fs.promises.access(blacklistFilePath, fs.constants.F_OK);
} catch (err) {
await fs.promises.writeFile(blacklistFilePath, JSON.stringify({}), 'utf8');
}
const blacklistData = fs.readFileSync(blacklistFilePath, 'utf8');
const blacklistData = await fs.promises.readFile(blacklistFilePath, 'utf8');
const blacklist = JSON.parse(blacklistData || '{}');
const info = blacklist[ip];
if (info && info.permanent) {
return true;
} else if (info) {
// 移除非法
// Remove invalid
delete blacklist[ip];
fs.writeFileSync(blacklistFilePath, JSON.stringify(blacklist, null, 4), 'utf8');
await fs.promises.writeFile(blacklistFilePath, JSON.stringify(blacklist, null, 4), 'utf8');
return false;
}
return false;
} catch (err) {
console.error(`读取黑名单文件出错:${err.message}`);
sysLogger.error(`Error reading blacklist file: ${err.message}`);
return false;
}
});
}
// 将IP添加到黑名单
// Add IP to blacklist
async addToBlacklist(ip) {
await this.blacklistMutex.runExclusive(async () => {
try {
if (!fs.existsSync(blacklistFilePath)) {
fs.writeFileSync(blacklistFilePath, JSON.stringify({}), 'utf8');
try {
await fs.promises.access(blacklistFilePath, fs.constants.F_OK);
} catch (err) {
await fs.promises.writeFile(blacklistFilePath, JSON.stringify({}), 'utf8');
}
const dataRaw = fs.readFileSync(blacklistFilePath, 'utf8');
const dataRaw = await fs.promises.readFile(blacklistFilePath, 'utf8');
const blacklist = JSON.parse(dataRaw || '{}');
if (!blacklist[ip]) {
blacklist[ip] = {permanent: true};
fs.writeFileSync(
await fs.promises.writeFile(
blacklistFilePath,
JSON.stringify(blacklist, null, 4),
'utf8'
);
}
} catch (err) {
console.error(`写入黑名单文件出错:${err.message}`);
sysLogger.error(`Error writing blacklist file: ${err.message}`);
}
});
}
// 检查是否有临时限制
// Check for temp limits
_hasTemporaryLimit(ip) {
const now = Date.now();
if (this.temporaryLimits[ip] && now < this.temporaryLimits[ip]) {
@@ -203,7 +186,7 @@ class RequestLogger {
return false;
}
// 获取剩余限制时间
// Get remaining limit time
_getRemainingTime(ip) {
const now = Date.now();
if (this.temporaryLimits[ip] && now < this.temporaryLimits[ip]) {
@@ -212,45 +195,45 @@ class RequestLogger {
const mm = Math.floor((remaining % 3600000) / 60000);
const ss = Math.floor((remaining % 60000) / 1000);
if (hh > 0) {
return `${hh}小时${mm}${ss}`;
return `${hh}${mm}m ${ss}s`;
} else if (mm > 0) {
return `${mm}${ss}`;
return `${mm}m ${ss}s`;
} else {
return `${ss}`;
return `${ss}s`;
}
}
return '未知时间';
return 'Unknown time';
}
// 清理过期的请求记录
// Clean expired request records
_cleanUpRecords(ip) {
const now = Date.now();
// 请求记录保留24小时
// Request records kept for 24 hours
if (this.ipRequestRecords[ip]) {
this.ipRequestRecords[ip] = this.ipRequestRecords[ip].filter(
t => now - t <= 24 * 60 * 60 * 1000
);
}
// 30秒触发记录同样保留24小时
// 30-second trigger records also kept for 24 hours
if (this.ip30sTriggerCount[ip]) {
this.ip30sTriggerCount[ip] = this.ip30sTriggerCount[ip].filter(
t => now - t <= 24 * 60 * 60 * 1000
);
}
// 清理过期的临时限制
// Clean expired temp limits
if (this.temporaryLimits[ip] && now >= this.temporaryLimits[ip]) {
delete this.temporaryLimits[ip];
this.saveTempLimitsToFile();
}
}
// 定期清理长期未活动的IP数据
// Periodically clean inactive IP data
cleanUpInactiveIPs() {
for (const ip in this.ipRequestRecords) {
const noRecentRequests = !this.ipRequestRecords[ip] || this.ipRequestRecords[ip].length === 0;
const no30sTriggers = !this.ip30sTriggerCount[ip] || this.ip30sTriggerCount[ip].length === 0;
// 若该IP无请求记录、无30秒触发记录且无临时限制
// If this IP has no request record, no 30s trigger, and no temp limit
if (noRecentRequests && no30sTriggers && !this._hasTemporaryLimit(ip)) {
delete this.ipRequestRecords[ip];
delete this.ip30sTriggerCount[ip];
@@ -259,10 +242,10 @@ class RequestLogger {
}
}
// 记录请求并检测
// Record request and detect
async logRequest({time, ip, location, model, session}) {
if (ENABLE_DETECTION && await this.isBlacklisted(ip)) {
throw new Error(`您已被永久限制。`);
throw new Error(`You have been permanently limited.`);
}
const baseInfo = `IP: ${ip} | Location: ${location} | Model: ${model} | Session: ${session}`;
if (!ENABLE_DETECTION) {
@@ -278,31 +261,31 @@ class RequestLogger {
await ipMutex.runExclusive(() => {
const now = Date.now();
// 检查临时限制
// Check temp limits
if (this._hasTemporaryLimit(ip)) {
const rt = this._getRemainingTime(ip);
limitInfo = ` | Limit: ${this.temporaryLimits[ip] - now}ms`;
throw new Error(`您被限制访问,请在${rt}后再试。`);
throw new Error(`You are rate limited, please try again after ${rt}.`);
}
// 记录请求 & 清理
// Record request & cleanup
if (!this.ipRequestRecords[ip]) {
this.ipRequestRecords[ip] = [];
}
this.ipRequestRecords[ip].push(now);
this._cleanUpRecords(ip);
// >=4次 → 加入黑名单
// >=4 times -> Add to blacklist
const requestsIn6s = this.ipRequestRecords[ip].filter(
t => now - t <= 6 * 1000
);
if (requestsIn6s.length >= 3) {
this.addToBlacklist(ip);
limitInfo = ' | Limit: PERMANENT';
throw new Error(`并发请求过多,已永久限制。`);
throw new Error(`Too many concurrent requests, permanently limited.`);
}
// (>=3次 → 临时限制)
// (>=3 times -> Temporary limit)
const requestsIn30s = this.ipRequestRecords[ip].filter(
t => now - t <= 30 * 1000
);
@@ -313,19 +296,19 @@ class RequestLogger {
this.ip30sTriggerCount[ip].push(now);
this._cleanUpRecords(ip);
// 24小时内触发次数 >= 5 → 限制6小时
// Triggered >=5 times in 24h -> limit 6 hours
if (this.ip30sTriggerCount[ip].length >= 5) {
const duration = 6 * 60 * 60 * 1000;
this.temporaryLimits[ip] = now + duration;
limitInfo = ` | Limit: ${duration}ms`;
this.saveTempLimitsToFile();
throw new Error(`您在24小时内多次频繁请求,限制6小时`);
throw new Error(`Frequent requests in 24 hours, limited for 6 hours`);
} else {
const duration = 60 * 1000;
this.temporaryLimits[ip] = now + duration;
limitInfo = ` | Limit: ${duration}ms`;
this.saveTempLimitsToFile();
throw new Error(`请求过多, 限制1分钟`);
throw new Error(`Too many requests, limited for 1 minute`);
}
}
});
File diff suppressed because it is too large Load Diff
@@ -1,115 +1,116 @@
import os from 'os';
import fs from 'fs';
import path from 'path';
import {execSync} from 'child_process';
export function detectBrowser(preferredBrowser = 'auto') {
const platform = os.platform();
let browsers = {
'chrome': null,
'edge': null
};
if (platform === 'win32') {
browsers.chrome = findWindowsBrowser('Chrome');
browsers.edge = findWindowsBrowser('Edge');
} else if (platform === 'darwin') {
browsers.chrome = findMacOSBrowser('Google Chrome');
browsers.edge = findMacOSBrowser('Microsoft Edge');
} else if (platform === 'linux') {
browsers.chrome = findLinuxBrowser('google-chrome');
//Arch下AUR安装的chrome为google-chrome-stable
if (browsers.chrome == null) {
browsers.chrome = findLinuxBrowser('google-chrome-stable');
}
browsers.edge = findLinuxBrowser('microsoft-edge');
}
if (preferredBrowser === 'auto' || preferredBrowser === undefined) {
if (browsers.chrome) {
return browsers.chrome;
} else if (browsers.edge) {
return browsers.edge;
}
} else if (browsers[preferredBrowser]) {
console.log(`使用${preferredBrowser === 'chrome' ? 'Chrome' : 'Edge'}浏览器`);
return browsers[preferredBrowser];
}
console.error('未找到Chrome或Edge浏览器,请确保已安装其中之一');
process.exit(1);
}
function findWindowsBrowser(browserName) {
const regKeys = {
'Chrome': ['chrome.exe', 'Google\\Chrome'],
'Edge': ['msedge.exe', 'Microsoft\\Edge']
};
const [exeName, folderName] = regKeys[browserName];
const regQuery = (key) => {
try {
return execSync(`reg query "${key}" /ve`).toString().trim().split('\r\n').pop().split(' ').pop();
} catch (error) {
return null;
}
};
let browserPath = regQuery(`HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\App Paths\\${exeName}`) ||
regQuery(`HKEY_CURRENT_USER\\Software\\Microsoft\\Windows\\CurrentVersion\\App Paths\\${exeName}`);
if (browserPath && fs.existsSync(browserPath)) {
return browserPath;
}
const commonPaths = [
`C:\\Program Files\\${browserName}\\Application\\${exeName}`,
`C:\\Program Files (x86)\\${browserName}\\Application\\${exeName}`,
`C:\\Program Files (x86)\\Microsoft\\${browserName}\\Application\\${exeName}`,
`${process.env.LOCALAPPDATA}\\${browserName}\\Application\\${exeName}`,
`${process.env.USERPROFILE}\\AppData\\Local\\${browserName}\\Application\\${exeName}`,
];
const foundPath = commonPaths.find(path => fs.existsSync(path));
if (foundPath) {
return foundPath;
}
const userAppDataPath = process.env.LOCALAPPDATA || `${process.env.USERPROFILE}\\AppData\\Local`;
const appDataPath = path.join(userAppDataPath, folderName, 'Application');
if (fs.existsSync(appDataPath)) {
const files = fs.readdirSync(appDataPath);
const exePath = files.find(file => file.toLowerCase() === exeName.toLowerCase());
if (exePath) {
return path.join(appDataPath, exePath);
}
}
return null;
}
function findMacOSBrowser(browserName) {
const paths = [
`/Applications/${browserName}.app/Contents/MacOS/${browserName}`,
`${os.homedir()}/Applications/${browserName}.app/Contents/MacOS/${browserName}`,
];
for (const path of paths) {
if (fs.existsSync(path)) {
return path;
}
}
return null;
}
function findLinuxBrowser(browserName) {
try {
return execSync(`which ${browserName}`).toString().trim();
} catch (error) {
return null;
}
import os from 'os';
import fs from 'fs';
import path from 'path';
import {execSync} from 'child_process';
import sysLogger from './sysLogger.mjs';
export function detectBrowser(preferredBrowser = 'auto') {
const platform = os.platform();
let browsers = {
'chrome': null,
'edge': null
};
if (platform === 'win32') {
browsers.chrome = findWindowsBrowser('Chrome');
browsers.edge = findWindowsBrowser('Edge');
} else if (platform === 'darwin') {
browsers.chrome = findMacOSBrowser('Google Chrome');
browsers.edge = findMacOSBrowser('Microsoft Edge');
} else if (platform === 'linux') {
browsers.chrome = findLinuxBrowser('google-chrome');
//Chrome installed via AUR on Arch is google-chrome-stable
if (browsers.chrome == null) {
browsers.chrome = findLinuxBrowser('google-chrome-stable');
}
browsers.edge = findLinuxBrowser('microsoft-edge');
}
if (preferredBrowser === 'auto' || preferredBrowser === undefined) {
if (browsers.chrome) {
return browsers.chrome;
} else if (browsers.edge) {
return browsers.edge;
}
} else if (browsers[preferredBrowser]) {
sysLogger.debug(`${preferredBrowser === 'chrome' ? 'Chrome' : 'Edge'}Browser`);
return browsers[preferredBrowser];
}
sysLogger.error('ChromeEdgeBrowser');
process.exit(1);
}
function findWindowsBrowser(browserName) {
const regKeys = {
'Chrome': ['chrome.exe', 'Google\\Chrome'],
'Edge': ['msedge.exe', 'Microsoft\\Edge']
};
const [exeName, folderName] = regKeys[browserName];
const regQuery = (key) => {
try {
return execSync(`reg query "${key}" /ve`).toString().trim().split('\r\n').pop().split(' ').pop();
} catch (error) {
return null;
}
};
let browserPath = regQuery(`HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\App Paths\\${exeName}`) ||
regQuery(`HKEY_CURRENT_USER\\Software\\Microsoft\\Windows\\CurrentVersion\\App Paths\\${exeName}`);
if (browserPath && fs.existsSync(browserPath)) {
return browserPath;
}
const commonPaths = [
`C:\\Program Files\\${browserName}\\Application\\${exeName}`,
`C:\\Program Files (x86)\\${browserName}\\Application\\${exeName}`,
`C:\\Program Files (x86)\\Microsoft\\${browserName}\\Application\\${exeName}`,
`${process.env.LOCALAPPDATA}\\${browserName}\\Application\\${exeName}`,
`${process.env.USERPROFILE}\\AppData\\Local\\${browserName}\\Application\\${exeName}`,
];
const foundPath = commonPaths.find(path => fs.existsSync(path));
if (foundPath) {
return foundPath;
}
const userAppDataPath = process.env.LOCALAPPDATA || `${process.env.USERPROFILE}\\AppData\\Local`;
const appDataPath = path.join(userAppDataPath, folderName, 'Application');
if (fs.existsSync(appDataPath)) {
const files = fs.readdirSync(appDataPath);
const exePath = files.find(file => file.toLowerCase() === exeName.toLowerCase());
if (exePath) {
return path.join(appDataPath, exePath);
}
}
return null;
}
function findMacOSBrowser(browserName) {
const paths = [
`/Applications/${browserName}.app/Contents/MacOS/${browserName}`,
`${os.homedir()}/Applications/${browserName}.app/Contents/MacOS/${browserName}`,
];
for (const path of paths) {
if (fs.existsSync(path)) {
return path;
}
}
return null;
}
function findLinuxBrowser(browserName) {
try {
return execSync(`which ${browserName}`).toString().trim();
} catch (error) {
return null;
}
}
@@ -1,206 +1,207 @@
/**
* @param {Object} page - Puppeteer
* @param {Object} options - 配置选项
* @param {number} options.width - 视口宽度
* @param {number} options.height - 视口高度
* @param {number} options.deviceScaleFactor - 设备缩放因子
* @param {boolean} options.isMobile - 模拟移动设备
* @param {boolean} options.hasTouch - 支持触摸
* @param {boolean} options.isLandscape - 横屏
* @returns {Promise<void>}
*/
export async function fixBrowserDisplay(page, options = {}) {
if (!page) {
console.error('页面对象为空,无法修复显示');
return;
}
const defaultOptions = {
width: 1280,
height: 800,
deviceScaleFactor: 1,
isMobile: false,
hasTouch: false,
isLandscape: true
};
const settings = {...defaultOptions, ...options};
try {
// 设置视口大小和设备比例
await page.setViewport({
width: settings.width,
height: settings.height,
deviceScaleFactor: settings.deviceScaleFactor,
isMobile: settings.isMobile,
hasTouch: settings.hasTouch,
isLandscape: settings.isLandscape
});
// 尝试调整窗口大小
const session = await page.target().createCDPSession();
await session.send('Emulation.setDeviceMetricsOverride', {
width: settings.width,
height: settings.height,
deviceScaleFactor: settings.deviceScaleFactor,
mobile: settings.isMobile,
screenWidth: settings.width,
screenHeight: settings.height
});
// 重置页面缩放
await page.evaluate(() => {
document.body.style.zoom = '100%';
document.body.style.transform = 'scale(1)';
document.body.style.transformOrigin = '0 0';
// 尝试修复可能存在的CSS
const styleElement = document.createElement('style');
styleElement.textContent = `
html, body {
width: 100% !important;
height: 100% !important;
overflow: auto !important;
}
.container, .main, #app, #root {
max-width: 100% !important;
width: auto !important;
}
`;
document.head.appendChild(styleElement);
window.dispatchEvent(new Event('resize'));
});
} catch (error) {
console.error('修复浏览器显示时出错:', error);
}
}
/**
* 调整CSS比例
* @param {Object} page - Puppeteer
* @param {number} scale - 缩放比例
* @returns {Promise<void>}
*/
export async function adjustCssScaling(page, scale = 1) {
if (!page) return;
try {
await page.evaluate((scale) => {
const styleElem = document.createElement('style');
styleElem.id = 'puppeteer-display-fix';
styleElem.textContent = `
html {
transform: scale(${scale});
transform-origin: top left;
width: ${100 / scale}% !important;
height: ${100 / scale}% !important;
}
`;
document.head.appendChild(styleElem);
// 重新计算布局
window.dispatchEvent(new Event('resize'));
}, scale);
} catch (error) {
console.error('调整CSS比例时出错:', error);
}
}
/**
* 修复高DPI
* @param {Object} page - Puppeteer
* @returns {Promise<void>}
*/
export async function fixHighDpiDisplay(page) {
if (!page) return;
try {
// 检测设备像素比
const devicePixelRatio = await page.evaluate(() => window.devicePixelRatio);
if (devicePixelRatio > 1) {
await page.setViewport({
width: 1280,
height: 800,
deviceScaleFactor: devicePixelRatio
});
await page.evaluate((dpr) => {
const meta = document.createElement('meta');
meta.setAttribute('name', 'viewport');
meta.setAttribute('content', `initial-scale=1, minimum-scale=1, maximum-scale=1, width=device-width, height=device-height, target-densitydpi=device-dpi, user-scalable=no`);
document.head.appendChild(meta);
}, devicePixelRatio);
}
} catch (error) {
console.error('修复高DPI显示时出错:', error);
}
}
/**
* 完整浏览器显示优化
* @param {Object} page - Puppeteer
* @param {Object} options - 配置
* @returns {Promise<void>}
*/
export async function optimizeBrowserDisplay(page, options = {}) {
const defaultOptions = {
width: 1280,
height: 800,
deviceScaleFactor: 1,
cssScale: null,
fixHighDpi: true,
forceResize: true
};
const config = {...defaultOptions, ...options};
try {
// 基本显示修复
await fixBrowserDisplay(page, {
width: config.width,
height: config.height,
deviceScaleFactor: config.deviceScaleFactor
});
// 修复高DPI显示
if (config.fixHighDpi) {
await fixHighDpiDisplay(page);
}
if (config.cssScale !== null) {
await adjustCssScaling(page, config.cssScale);
}
// 如果强制调整窗口大小
if (config.forceResize && !config.isHeadless) {
try {
const client = await page.target().createCDPSession();
await client.send('Browser.getWindowForTarget');
await client.send('Browser.setWindowBounds', {
windowId: 1,
bounds: {
width: config.width,
height: config.height
}
});
} catch (resizeError) {
// console.log('无法调整窗口大小:', resizeError.message);
try {
await page.evaluate((width, height) => {
window.resizeTo(width, height);
}, config.width, config.height);
} catch (altError) {
console.log('失败:', altError.message);
}
}
}
} catch (error) {
console.error('浏览器显示优化失败:', error);
}
}
import sysLogger from './sysLogger.mjs';
/**
* @param {Object} page - Puppeteer
* @param {Object} options -
* @param {number} options.width -
* @param {number} options.height -
* @param {number} options.deviceScaleFactor -
* @param {boolean} options.isMobile -
* @param {boolean} options.hasTouch -
* @param {boolean} options.isLandscape -
* @returns {Promise<void>}
*/
export async function fixBrowserDisplay(page, options = {}) {
if (!page) {
sysLogger.error('Page object is empty, unable to fix display');
return;
}
const defaultOptions = {
width: 1280,
height: 800,
deviceScaleFactor: 1,
isMobile: false,
hasTouch: false,
isLandscape: true
};
const settings = {...defaultOptions, ...options};
try {
// Set viewport size and device ratio
await page.setViewport({
width: settings.width,
height: settings.height,
deviceScaleFactor: settings.deviceScaleFactor,
isMobile: settings.isMobile,
hasTouch: settings.hasTouch,
isLandscape: settings.isLandscape
});
// Try resizing window
const session = await page.target().createCDPSession();
await session.send('Emulation.setDeviceMetricsOverride', {
width: settings.width,
height: settings.height,
deviceScaleFactor: settings.deviceScaleFactor,
mobile: settings.isMobile,
screenWidth: settings.width,
screenHeight: settings.height
});
// Reset page zoom
await page.evaluate(() => {
document.body.style.zoom = '100%';
document.body.style.transform = 'scale(1)';
document.body.style.transformOrigin = '0 0';
// Attempt to fix possible CSS
const styleElement = document.createElement('style');
styleElement.textContent = `
html, body {
width: 100% !important;
height: 100% !important;
overflow: auto !important;
}
.container, .main, #app, #root {
max-width: 100% !important;
width: auto !important;
}
`;
document.head.appendChild(styleElement);
window.dispatchEvent(new Event('resize'));
});
} catch (error) {
sysLogger.error('Browser:', error);
}
}
/**
* CSS
* @param {Object} page - Puppeteer
* @param {number} scale -
* @returns {Promise<void>}
*/
export async function adjustCssScaling(page, scale = 1) {
if (!page) return;
try {
await page.evaluate((scale) => {
const styleElem = document.createElement('style');
styleElem.id = 'puppeteer-display-fix';
styleElem.textContent = `
html {
transform: scale(${scale});
transform-origin: top left;
width: ${100 / scale}% !important;
height: ${100 / scale}% !important;
}
`;
document.head.appendChild(styleElem);
// Recalculate layout
window.dispatchEvent(new Event('resize'));
}, scale);
} catch (error) {
sysLogger.error('Error adjusting CSS ratio:', error);
}
}
/**
* DPI
* @param {Object} page - Puppeteer
* @returns {Promise<void>}
*/
export async function fixHighDpiDisplay(page) {
if (!page) return;
try {
// Detect device pixel ratio
const devicePixelRatio = await page.evaluate(() => window.devicePixelRatio);
if (devicePixelRatio > 1) {
await page.setViewport({
width: 1280,
height: 800,
deviceScaleFactor: devicePixelRatio
});
await page.evaluate((dpr) => {
const meta = document.createElement('meta');
meta.setAttribute('name', 'viewport');
meta.setAttribute('content', `initial-scale=1, minimum-scale=1, maximum-scale=1, width=device-width, height=device-height, target-densitydpi=device-dpi, user-scalable=no`);
document.head.appendChild(meta);
}, devicePixelRatio);
}
} catch (error) {
sysLogger.error('Fix High-DPI display:', error);
}
}
/**
* Browser
* @param {Object} page - Puppeteer
* @param {Object} options -
* @returns {Promise<void>}
*/
export async function optimizeBrowserDisplay(page, options = {}) {
const defaultOptions = {
width: 1280,
height: 800,
deviceScaleFactor: 1,
cssScale: null,
fixHighDpi: true,
forceResize: true
};
const config = {...defaultOptions, ...options};
try {
// Basic display fix
await fixBrowserDisplay(page, {
width: config.width,
height: config.height,
deviceScaleFactor: config.deviceScaleFactor
});
// Fix High-DPI display
if (config.fixHighDpi) {
await fixHighDpiDisplay(page);
}
if (config.cssScale !== null) {
await adjustCssScaling(page, config.cssScale);
}
// If forced to resize window
if (config.forceResize && !config.isHeadless) {
try {
const client = await page.target().createCDPSession();
await client.send('Browser.getWindowForTarget');
await client.send('Browser.setWindowBounds', {
windowId: 1,
bounds: {
width: config.width,
height: config.height
}
});
} catch (resizeError) {
// sysLogger.debug('Unable to resize window:', resizeError.message);
try {
await page.evaluate((width, height) => {
window.resizeTo(width, height);
}, config.width, config.height);
} catch (altError) {
sysLogger.debug('Failed:', altError.message);
}
}
}
} catch (error) {
sysLogger.error('BrowserDisplay optimization failed:', error);
}
}
File diff suppressed because it is too large Load Diff
@@ -1,242 +1,210 @@
import * as docx from "docx";
import cookie from "cookie";
import fs from "fs";
import {execSync} from "child_process";
function getGitRevision() {
// get git revision and branch
try {
const revision = execSync("git rev-parse --short HEAD", {stdio: "pipe"}).toString().trim();
const branch = execSync("git rev-parse --abbrev-ref HEAD", {stdio: "pipe"}).toString().trim();
return {revision, branch};
} catch (e) {
return {revision: "unknown", branch: "unknown"};
}
}
// 创建目录
function createDirectoryIfNotExists(dirPath) {
if (!fs.existsSync(dirPath)) {
fs.mkdirSync(dirPath, {recursive: true});
}
}
function extractCookie(cookies) {
let jwtSession = null;
let jwtToken = null;
let ds = null;
let dsr = null;
let you_subscription = null;
let youpro_subscription = null;
cookies = cookie.parse(cookies);
if (cookies["stytch_session"]) jwtSession = cookies["stytch_session"];
if (cookies["stytch_session_jwt"]) jwtToken = cookies["stytch_session_jwt"];
if (cookies["DS"]) ds = cookies["DS"];
if (cookies["DSR"]) dsr = cookies["DSR"];
if (cookies["you_subscription"]) you_subscription = cookies["you_subscription"];
if (cookies["youpro_subscription"]) youpro_subscription = cookies["youpro_subscription"];
return {jwtSession, jwtToken, ds, dsr, you_subscription, youpro_subscription};
}
function getSessionCookie(jwtSession, jwtToken, ds, dsr, you_subscription, youpro_subscription) {
let sessionCookie = [];
// 处理旧版 cookie
if (jwtSession && jwtToken) {
sessionCookie = [
{
name: "stytch_session",
value: jwtSession,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: false,
secure: true,
sameSite: "Lax",
},
{
name: "ydc_stytch_session",
value: jwtSession,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: true,
secure: true,
sameSite: "Lax",
},
{
name: "stytch_session_jwt",
value: jwtToken,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: false,
secure: true,
sameSite: "Lax",
},
{
name: "ydc_stytch_session_jwt",
value: jwtToken,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: true,
secure: true,
sameSite: "Lax",
}
];
}
// 处理新版 cookie
if (ds) {
sessionCookie.push({
name: "DS",
value: ds,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: false,
secure: true,
sameSite: "Lax",
});
}
if (dsr) {
sessionCookie.push({
name: "DSR",
value: dsr,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: false,
secure: true,
sameSite: "Lax",
});
}
if (you_subscription) {
sessionCookie.push({
name: "you_subscription",
value: you_subscription,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: false,
secure: true,
sameSite: "Lax",
});
}
if (youpro_subscription) {
sessionCookie.push({
name: "youpro_subscription",
value: youpro_subscription,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: false,
secure: true,
sameSite: "Lax",
});
}
// 添加隐身模式 cookie(如果启用)
if (process.env.INCOGNITO_MODE === "true") {
sessionCookie.push({
name: "incognito",
value: "true",
domain: "you.com",
path: "/",
expires: 1800000000,
secure: true,
});
}
return sessionCookie;
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function createDocx(content) {
let paragraphs = [];
content.split("\n").forEach((line) => {
paragraphs.push(
new docx.Paragraph({
children: [new docx.TextRun(line)],
})
);
});
let doc = new docx.Document({
sections: [
{
properties: {},
children: paragraphs,
},
],
});
return docx.Packer.toBuffer(doc).then((buffer) => buffer);
}
// eventStream util
function createEvent(event, data) {
// if data is object, stringify it
if (typeof data === "object") {
data = JSON.stringify(data);
}
return `event: ${event}\ndata: ${data}\n\n`;
}
function extractPerplexityCookie(cookieString) {
const cookies = cookie.parse(cookieString);
return {
sessionToken: cookies['__Secure-next-auth.session-token'],
isIncognito: cookies['pplx.is-incognito'] === 'true'
};
}
function getPerplexitySessionCookie(extractedCookie) {
let sessionCookie = [];
if (extractedCookie.sessionToken) {
sessionCookie.push({
name: "__Secure-next-auth.session-token",
value: extractedCookie.sessionToken,
domain: "www.perplexity.ai",
path: "/",
expires: 1800000000,
httpOnly: true,
secure: true,
sameSite: "Lax",
});
}
// 添加无痕模式 cookie (如果启用)
if (process.env.INCOGNITO_MODE === "true") {
sessionCookie.push({
name: "pplx.is-incognito",
value: "true",
domain: "www.perplexity.ai",
path: "/",
expires: 1800000000,
httpOnly: false,
secure: true,
sameSite: "Lax",
});
}
return sessionCookie;
}
export {
createEvent,
createDirectoryIfNotExists,
sleep,
extractCookie,
getSessionCookie,
createDocx,
getGitRevision,
extractPerplexityCookie,
getPerplexitySessionCookie
import * as docx from "docx";
import cookie from "cookie";
import fs from "fs";
import {execSync} from "child_process";
import sysLogger from './sysLogger.mjs';
let cachedGitRevision = null;
function getGitRevision() {
if (cachedGitRevision) {
return cachedGitRevision;
}
// get git revision and branch
try {
const revision = execSync("git rev-parse --short HEAD", {stdio: "pipe"}).toString().trim();
const branch = execSync("git rev-parse --abbrev-ref HEAD", {stdio: "pipe"}).toString().trim();
cachedGitRevision = {revision, branch};
return cachedGitRevision;
} catch (e) {
cachedGitRevision = {revision: "unknown", branch: "unknown"};
return cachedGitRevision;
}
}
// Create directory
function createDirectoryIfNotExists(dirPath) {
if (!fs.existsSync(dirPath)) {
fs.mkdirSync(dirPath, {recursive: true});
}
}
function extractCookie(cookies) {
let jwtSession = null;
let jwtToken = null;
let ds = null;
let dsr = null;
let you_subscription = null;
let youpro_subscription = null;
cookies = cookie.parse(cookies);
if (cookies["stytch_session"]) jwtSession = cookies["stytch_session"];
if (cookies["stytch_session_jwt"]) jwtToken = cookies["stytch_session_jwt"];
if (cookies["DS"]) ds = cookies["DS"];
if (cookies["DSR"]) dsr = cookies["DSR"];
if (cookies["you_subscription"]) you_subscription = cookies["you_subscription"];
if (cookies["youpro_subscription"]) youpro_subscription = cookies["youpro_subscription"];
return {jwtSession, jwtToken, ds, dsr, you_subscription, youpro_subscription};
}
function getSessionCookie(jwtSession, jwtToken, ds, dsr, you_subscription, youpro_subscription) {
let sessionCookie = [];
// Process legacy cookie
if (jwtSession && jwtToken) {
sessionCookie = [
{
name: "stytch_session",
value: jwtSession,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: false,
secure: true,
sameSite: "Lax",
},
{
name: "ydc_stytch_session",
value: jwtSession,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: true,
secure: true,
sameSite: "Lax",
},
{
name: "stytch_session_jwt",
value: jwtToken,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: false,
secure: true,
sameSite: "Lax",
},
{
name: "ydc_stytch_session_jwt",
value: jwtToken,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: true,
secure: true,
sameSite: "Lax",
}
];
}
// Process new cookie
if (ds) {
sessionCookie.push({
name: "DS",
value: ds,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: false,
secure: true,
sameSite: "Lax",
});
}
if (dsr) {
sessionCookie.push({
name: "DSR",
value: dsr,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: false,
secure: true,
sameSite: "Lax",
});
}
if (you_subscription) {
sessionCookie.push({
name: "you_subscription",
value: you_subscription,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: false,
secure: true,
sameSite: "Lax",
});
}
if (youpro_subscription) {
sessionCookie.push({
name: "youpro_subscription",
value: youpro_subscription,
domain: "you.com",
path: "/",
expires: 1800000000,
httpOnly: false,
secure: true,
sameSite: "Lax",
});
}
// Add incognito mode cookie (if enabled)
if (process.env.INCOGNITO_MODE === "true") {
sessionCookie.push({
name: "incognito",
value: "true",
domain: "you.com",
path: "/",
expires: 1800000000,
secure: true,
});
}
return sessionCookie;
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function createDocx(content) {
let paragraphs = [];
content.split("\n").forEach((line) => {
paragraphs.push(
new docx.Paragraph({
children: [new docx.TextRun(line)],
})
);
});
let doc = new docx.Document({
sections: [
{
properties: {},
children: paragraphs,
},
],
});
return docx.Packer.toBuffer(doc).then((buffer) => buffer);
}
// eventStream util
function createEvent(event, data) {
// if data is object, stringify it
if (typeof data === "object") {
data = JSON.stringify(data);
}
if (event === "data") {
return `data: ${data}\n\n`;
}
return `event: ${event}\ndata: ${data}\n\n`;
}
export {
createEvent,
createDirectoryIfNotExists,
sleep,
extractCookie,
getSessionCookie,
createDocx,
getGitRevision,
};
@@ -1,153 +1,154 @@
import {exec, execSync} from 'child_process';
import {promisify} from 'util';
import fs from 'fs';
import path from 'path';
import os from 'os';
import {fileURLToPath} from 'url';
import {setupBrowserFingerprint} from './browserFingerprint.mjs';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const execPromise = promisify(exec);
/**
* @param {string} userDataDir - 用户数据目录
* @param {string} edgePath - Edge浏览器路径
* @param {number} debugPort - 调试端口
* @returns {Promise<object>} - browser和page
*/
export async function launchEdgeBrowser(userDataDir, edgePath, debugPort = 9222) {
if (!fs.existsSync(userDataDir)) {
fs.mkdirSync(userDataDir, {recursive: true});
}
// 关闭可能的Edge进程
try {
if (os.platform() === 'win32') {
await execPromise('taskkill /f /im msedge.exe').catch(() => {
});
} else if (os.platform() === 'darwin') {
// macOS
await execPromise('pkill -f "Microsoft Edge"').catch(() => {
});
} else {
// Linux
await execPromise('pkill -f "microsoft-edge"').catch(() => {
});
}
} catch (e) {
}
const remoteDebuggingPort = debugPort;
let edgeProcess;
const args = [
`--remote-debugging-port=${remoteDebuggingPort}`,
`--user-data-dir="${userDataDir}"`,
'--no-first-run',
'--no-default-browser-check',
'--disable-popup-blocking',
'--disable-infobars',
'--disable-translate',
'--disable-sync',
'--window-size=1280,850',
'--force-device-scale-factor=1',
'about:blank' // 打开空白页
];
try {
// 启动Edge浏览器
const cmdArgs = args.join(' ');
const cmd = `"${edgePath}" ${cmdArgs}`;
edgeProcess = exec(cmd);
console.log(`等待Edge浏览器启动...`);
await new Promise(resolve => setTimeout(resolve, 3000));
const puppeteer = await import('puppeteer-core');
// 连接到浏览器
const browser = await puppeteer.connect({
browserURL: `http://127.0.0.1:${remoteDebuggingPort}`,
defaultViewport: {width: 1280, height: 850}
});
// 获取第一个页面
const pages = await browser.pages();
let page = pages[0];
if (!page) {
page = await browser.newPage();
}
const originalUserAgent = await page.evaluate(() => navigator.userAgent);
// console.log(`Edge浏览器原始用户代理: ${originalUserAgent}`);
// 应用随机指纹
const fingerprint = await setupBrowserFingerprint(page, 'edge');
// 验证指纹是否成功应用
const newUserAgent = await page.evaluate(() => navigator.userAgent);
// console.log(`Edge浏览器应用指纹后用户代理: ${newUserAgent}`);
if (!newUserAgent.includes('Edg')) {
console.warn(`警告: 浏览器可能不是Edge。`);
}
return {
browser,
page,
process: edgeProcess,
fingerprint: fingerprint
};
} catch (error) {
console.error(`启动Edge浏览器失败:`, error);
if (edgeProcess) {
try {
edgeProcess.kill();
} catch (e) {
}
}
throw error;
}
}
/**
* 查找Edge浏览器路径
* @returns {string|null}
*/
export function findEdgePath() {
const platform = os.platform();
if (platform === 'win32') {
const commonPaths = [
`${process.env['ProgramFiles(x86)']}\\Microsoft\\Edge\\Application\\msedge.exe`,
`${process.env.ProgramFiles}\\Microsoft\\Edge\\Application\\msedge.exe`,
`${process.env.LOCALAPPDATA}\\Microsoft\\Edge\\Application\\msedge.exe`,
`C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe`,
`C:\\Program Files\\Microsoft\\Edge\\Application\\msedge.exe`
];
for (const path of commonPaths) {
if (fs.existsSync(path)) {
return path;
}
}
} else if (platform === 'darwin') {
const macPath = '/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge';
if (fs.existsSync(macPath)) {
return macPath;
}
} else {
// Linux
try {
const {stdout} = execSync('which microsoft-edge');
if (stdout && stdout.trim()) {
return stdout.trim();
}
} catch (e) {
}
}
return null;
import {exec, execSync} from 'child_process';
import {promisify} from 'util';
import fs from 'fs';
import path from 'path';
import os from 'os';
import {fileURLToPath} from 'url';
import {setupBrowserFingerprint} from './browserFingerprint.mjs';
import sysLogger from './sysLogger.mjs';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const execPromise = promisify(exec);
/**
* @param {string} userDataDir -
* @param {string} edgePath - EdgeBrowser
* @param {number} debugPort -
* @returns {Promise<object>} - browserpage
*/
export async function launchEdgeBrowser(userDataDir, edgePath, debugPort = 9222) {
if (!fs.existsSync(userDataDir)) {
fs.mkdirSync(userDataDir, {recursive: true});
}
// Close possible Edge processes
try {
if (os.platform() === 'win32') {
await execPromise('taskkill /f /im msedge.exe').catch(() => {
});
} else if (os.platform() === 'darwin') {
// macOS
await execPromise('pkill -f "Microsoft Edge"').catch(() => {
});
} else {
// Linux
await execPromise('pkill -f "microsoft-edge"').catch(() => {
});
}
} catch (e) {
}
const remoteDebuggingPort = debugPort;
let edgeProcess;
const args = [
`--remote-debugging-port=${remoteDebuggingPort}`,
`--user-data-dir="${userDataDir}"`,
'--no-first-run',
'--no-default-browser-check',
'--disable-popup-blocking',
'--disable-infobars',
'--disable-translate',
'--disable-sync',
'--window-size=1280,850',
'--force-device-scale-factor=1',
'about:blank' // Open blank page
];
try {
// EdgeBrowser
const cmdArgs = args.join(' ');
const cmd = `"${edgePath}" ${cmdArgs}`;
edgeProcess = exec(cmd);
sysLogger.debug(`Waiting for Edge browser to start...`);
await new Promise(resolve => setTimeout(resolve, 3000));
const puppeteer = await import('puppeteer-core');
// Browser
const browser = await puppeteer.connect({
browserURL: `http://127.0.0.1:${remoteDebuggingPort}`,
defaultViewport: {width: 1280, height: 850}
});
// Get first page
const pages = await browser.pages();
let page = pages[0];
if (!page) {
page = await browser.newPage();
}
const originalUserAgent = await page.evaluate(() => navigator.userAgent);
// sysLogger.debug(`EdgeBrowser: ${originalUserAgent}`);
// Apply random fingerprint
const fingerprint = await setupBrowserFingerprint(page, 'edge');
// Verify if fingerprint was successfully applied
const newUserAgent = await page.evaluate(() => navigator.userAgent);
// sysLogger.debug(`EdgeBrowserApply fingerprint: ${newUserAgent}`);
if (!newUserAgent.includes('Edg')) {
sysLogger.warn(`: BrowserEdge。`);
}
return {
browser,
page,
process: edgeProcess,
fingerprint: fingerprint
};
} catch (error) {
sysLogger.error(`EdgeBrowserFailed:`, error);
if (edgeProcess) {
try {
edgeProcess.kill();
} catch (e) {
}
}
throw error;
}
}
/**
* EdgeBrowser
* @returns {string|null}
*/
export function findEdgePath() {
const platform = os.platform();
if (platform === 'win32') {
const commonPaths = [
`${process.env['ProgramFiles(x86)']}\\Microsoft\\Edge\\Application\\msedge.exe`,
`${process.env.ProgramFiles}\\Microsoft\\Edge\\Application\\msedge.exe`,
`${process.env.LOCALAPPDATA}\\Microsoft\\Edge\\Application\\msedge.exe`,
`C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe`,
`C:\\Program Files\\Microsoft\\Edge\\Application\\msedge.exe`
];
for (const path of commonPaths) {
if (fs.existsSync(path)) {
return path;
}
}
} else if (platform === 'darwin') {
const macPath = '/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge';
if (fs.existsSync(macPath)) {
return macPath;
}
} else {
// Linux
try {
const {stdout} = execSync('which microsoft-edge');
if (stdout && stdout.trim()) {
return stdout.trim();
}
} catch (e) {
}
}
return null;
}
+45
View File
@@ -0,0 +1,45 @@
const LOG_LEVELS = {
DEBUG: 0,
INFO: 1,
WARN: 2,
ERROR: 3,
};
const currentLevel = LOG_LEVELS[(process.env.LOG_LEVEL || 'info').toUpperCase()] ?? LOG_LEVELS.INFO;
const colors = {
reset: "\x1b[0m",
blue: "\x1b[34m",
yellow: "\x1b[33m",
red: "\x1b[31m",
gray: "\x1b[90m"
};
class SysLogger {
debug(...args) {
if (currentLevel <= LOG_LEVELS.DEBUG) {
console.log(colors.gray + "[DEBUG]" + colors.reset, ...args);
}
}
info(...args) {
if (currentLevel <= LOG_LEVELS.INFO) {
console.log(colors.blue + "[INFO]" + colors.reset, ...args);
}
}
warn(...args) {
if (currentLevel <= LOG_LEVELS.WARN) {
console.warn(colors.yellow + "[WARN]" + colors.reset, ...args);
}
}
error(...args) {
if (currentLevel <= LOG_LEVELS.ERROR) {
console.error(colors.red + "[ERROR]" + colors.reset, ...args);
}
}
}
const sysLogger = new SysLogger();
export default sysLogger;
@@ -1,204 +1,205 @@
import fs from "fs";
import path from "path";
import {fileURLToPath} from "url";
import {Mutex} from "async-mutex";
const configMutex = new Mutex(); // 互斥锁
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const CONFIG_FILE_PATH = path.join(__dirname, "../config.mjs");
// 仅在 USE_MANUAL_LOGIN 为 false 且 ENABLE_AUTO_COOKIE_UPDATE 为 true 时生效
const ENABLE_AUTO_COOKIE_UPDATE = process.env.ENABLE_AUTO_COOKIE_UPDATE === "true";
function unifyQuotesForJSON(str) {
// 正则匹配 `` `...` ``
let out = str.replace(/`([^`]*)`/g, (match, p1) => {
const safe = p1.replace(/"/g, '\\"');
return `"${safe}"`;
});
out = out.replace(/'([^']*)'/g, (match, p1) => {
const safe = p1.replace(/"/g, '\\"');
return `"${safe}"`;
});
return out;
}
/**
* cookies 解析出 DS DSR
* @param {Array} cookies 获取到的 cookie 数组
* @returns {{ ds?: string, dsr?: string }}
*/
function parseDSAndDSR(cookies) {
let dsValue, dsrValue;
for (const c of cookies) {
if (c.name === "DS") {
dsValue = c.value;
} else if (c.name === "DSR") {
dsrValue = c.value;
}
}
return {ds: dsValue, dsr: dsrValue};
}
/**
* DS 中解析 email 字段
* @param {string} dsToken DS cookie
* @returns {string|null} 返回 email或null
*/
function decodeEmailFromDs(dsToken) {
try {
const parts = dsToken.split(".");
if (parts.length < 2) return null;
const payload = JSON.parse(Buffer.from(parts[1], "base64").toString("utf-8"));
return payload.email || null;
} catch (err) {
return null;
}
}
/**
* cookie 数组转换 "name=value; name=value"
* @param {Array} cookies
* @returns {string}
*/
function cookiesToStringAll(cookies) {
return cookies.map(c => `${c.name}=${c.value}`).join("; ");
}
/**
* cookie 转换数组
* 每个对象如 { name, value }
* @param {string} cookieStr
* @returns {Array}
*/
function parseCookieString(cookieStr) {
return cookieStr.split("; ").map(entry => {
const [name, value] = entry.split("=", 2);
return {name, value};
});
}
/**
* 本地 configObj.sessions 查找与指定 email 匹配的 session
* @param {object} configObj 解析后 config
* @param {string} email 匹配的邮箱
* @returns {{ index: number, oldCookie: string, ds: string, dsr: string } | null}
*/
function findSessionByEmail(configObj, email) {
if (!Array.isArray(configObj.sessions)) return null;
for (let i = 0; i < configObj.sessions.length; i++) {
const cookieStr = configObj.sessions[i].cookie || "";
const dsMatch = /DS=([^;\s]+)/.exec(cookieStr);
if (!dsMatch) continue;
const dsValue = dsMatch[1];
const dsEmail = decodeEmailFromDs(dsValue);
if (dsEmail && dsEmail.toLowerCase() === email.toLowerCase()) {
const dsrMatch = /DSR=([^;\s]+)/.exec(cookieStr);
const dsrValue = dsrMatch ? dsrMatch[1] : "";
return {
index: i,
oldCookie: cookieStr,
ds: dsValue,
dsr: dsrValue
};
}
}
return null;
}
/**
* config.mjs 中匹配相同 email session DS DSR 有变化则更新整个 cookie
* @param {import('puppeteer-core').Page} page
*/
export async function updateLocalConfigCookieByEmail(page) {
if (!ENABLE_AUTO_COOKIE_UPDATE || process.env.USE_MANUAL_LOGIN === "true") {
return;
}
// 尝试从 “https://you.com/api/instrumentation” 获取 cookie
let cookieStringFromInstrumentation = "";
try {
const instrRequest = await page.waitForRequest(
req => req.url().includes("/api/instrumentation"),
{timeout: 5000}
);
if (instrRequest) {
cookieStringFromInstrumentation = instrRequest.headers()["cookie"];
}
} catch (err) {
}
let allCookiesString = "";
if (cookieStringFromInstrumentation) {
allCookiesString = cookieStringFromInstrumentation;
} else {
// 使用 page.cookies() 获取
const cookies = await page.cookies("https://you.com");
allCookiesString = cookiesToStringAll(cookies);
}
const cookieArray = parseCookieString(allCookiesString);
const {ds: newDs, dsr: newDsr} = parseDSAndDSR(cookieArray);
if (!newDs) {
console.log("网页未找到 DS,跳过更新。");
return;
}
const newEmail = decodeEmailFromDs(newDs);
if (!newEmail) {
console.log("[网页无法从 DS 解出 email,跳过更新。");
return;
}
// 互斥区
await configMutex.runExclusive(async () => {
try {
if (!fs.existsSync(CONFIG_FILE_PATH)) {
console.warn(`找不到 config.mjs: ${CONFIG_FILE_PATH}`);
return;
}
const raw = fs.readFileSync(CONFIG_FILE_PATH, "utf8");
// 去掉 export const config =
let jsonString = raw.replace(/^export const config\s*=\s*/, "").trim();
jsonString = unifyQuotesForJSON(jsonString);
const configObj = JSON.parse(jsonString);
const found = findSessionByEmail(configObj, newEmail);
if (!found) {
console.log(`未能在 config 中找到 email=${newEmail} 的 session,跳过更新。`);
return;
}
if (found.ds === newDs && found.dsr === newDsr) {
console.log(`DS/DSR 未变化(email=${newEmail}),不更新。`);
return;
}
configObj.sessions[found.index].cookie = allCookiesString;
const newFileContent = "export const config = " + JSON.stringify(configObj, null, 4);
fs.writeFileSync(CONFIG_FILE_PATH, newFileContent, "utf8");
console.log(`Cookie已更新(email=${newEmail})`);
} catch (err) {
console.warn("Cookie更新过程出错:", err);
}
});
}
/**
* 非阻塞
* @param {import('puppeteer-core').Page} page
*/
export function updateLocalConfigCookieByEmailNonBlocking(page) {
// 保证异步
setImmediate(() => {
updateLocalConfigCookieByEmail(page).catch(err =>
console.error("Cookie update error:", err)
);
});
import fs from "fs";
import path from "path";
import {fileURLToPath} from "url";
import {Mutex} from "async-mutex";
import sysLogger from '../utils/sysLogger.mjs';
const configMutex = new Mutex(); // Mutex lock
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const CONFIG_FILE_PATH = path.join(process.cwd(), "config.mjs");
// Only effective when USE_MANUAL_LOGIN is false and ENABLE_AUTO_COOKIE_UPDATE is true
const ENABLE_AUTO_COOKIE_UPDATE = process.env.ENABLE_AUTO_COOKIE_UPDATE === "true";
function unifyQuotesForJSON(str) {
// Regex match `` `...` ``
let out = str.replace(/`([^`]*)`/g, (match, p1) => {
const safe = p1.replace(/"/g, '\\"');
return `"${safe}"`;
});
out = out.replace(/'([^']*)'/g, (match, p1) => {
const safe = p1.replace(/"/g, '\\"');
return `"${safe}"`;
});
return out;
}
/**
* cookies DS DSR
* @param {Array} cookies cookie
* @returns {{ ds?: string, dsr?: string }}
*/
function parseDSAndDSR(cookies) {
let dsValue, dsrValue;
for (const c of cookies) {
if (c.name === "DS") {
dsValue = c.value;
} else if (c.name === "DSR") {
dsrValue = c.value;
}
}
return {ds: dsValue, dsr: dsrValue};
}
/**
* DS email
* @param {string} dsToken DS cookie
* @returns {string|null} emailnull
*/
function decodeEmailFromDs(dsToken) {
try {
const parts = dsToken.split(".");
if (parts.length < 2) return null;
const payload = JSON.parse(Buffer.from(parts[1], "base64").toString("utf-8"));
return payload.email || null;
} catch (err) {
return null;
}
}
/**
* cookie "name=value; name=value"
* @param {Array} cookies
* @returns {string}
*/
function cookiesToStringAll(cookies) {
return cookies.map(c => `${c.name}=${c.value}`).join("; ");
}
/**
* cookie
* { name, value }
* @param {string} cookieStr
* @returns {Array}
*/
function parseCookieString(cookieStr) {
return cookieStr.split("; ").map(entry => {
const [name, value] = entry.split("=", 2);
return {name, value};
});
}
/**
* configObj.sessions email session
* @param {object} configObj config
* @param {string} email
* @returns {{ index: number, oldCookie: string, ds: string, dsr: string } | null}
*/
function findSessionByEmail(configObj, email) {
if (!Array.isArray(configObj.sessions)) return null;
for (let i = 0; i < configObj.sessions.length; i++) {
const cookieStr = configObj.sessions[i].cookie || "";
const dsMatch = /DS=([^;\s]+)/.exec(cookieStr);
if (!dsMatch) continue;
const dsValue = dsMatch[1];
const dsEmail = decodeEmailFromDs(dsValue);
if (dsEmail && dsEmail.toLowerCase() === email.toLowerCase()) {
const dsrMatch = /DSR=([^;\s]+)/.exec(cookieStr);
const dsrValue = dsrMatch ? dsrMatch[1] : "";
return {
index: i,
oldCookie: cookieStr,
ds: dsValue,
dsr: dsrValue
};
}
}
return null;
}
/**
* config.mjs email session DS DSR cookie
* @param {import('puppeteer-core').Page} page
*/
export async function updateLocalConfigCookieByEmail(page) {
if (!ENABLE_AUTO_COOKIE_UPDATE || process.env.USE_MANUAL_LOGIN === "true") {
return;
}
// Try fetching cookie from https://you.com/api/instrumentation
let cookieStringFromInstrumentation = "";
try {
const instrRequest = await page.waitForRequest(
req => req.url().includes("/api/instrumentation"),
{timeout: 5000}
);
if (instrRequest) {
cookieStringFromInstrumentation = instrRequest.headers()["cookie"];
}
} catch (err) {
}
let allCookiesString = "";
if (cookieStringFromInstrumentation) {
allCookiesString = cookieStringFromInstrumentation;
} else {
// Fetch using page.cookies()
const cookies = await page.cookies("https://you.com");
allCookiesString = cookiesToStringAll(cookies);
}
const cookieArray = parseCookieString(allCookiesString);
const {ds: newDs, dsr: newDsr} = parseDSAndDSR(cookieArray);
if (!newDs) {
sysLogger.debug("DS not found on page, skipping update.");
return;
}
const newEmail = decodeEmailFromDs(newDs);
if (!newEmail) {
sysLogger.debug("[Page cannot resolve email from DS, skipping update.");
return;
}
// Mutex area
await configMutex.runExclusive(async () => {
try {
if (!fs.existsSync(CONFIG_FILE_PATH)) {
sysLogger.warn(`config.mjs not found: ${CONFIG_FILE_PATH}`);
return;
}
const raw = fs.readFileSync(CONFIG_FILE_PATH, "utf8");
// Remove export const config =
let jsonString = raw.replace(/^export const config\s*=\s*/, "").trim();
jsonString = unifyQuotesForJSON(jsonString);
const configObj = JSON.parse(jsonString);
const found = findSessionByEmail(configObj, newEmail);
if (!found) {
sysLogger.debug(`Could not find session for email=${newEmail} in config, skipping update.`);
return;
}
if (found.ds === newDs && found.dsr === newDsr) {
sysLogger.debug(`DS/DSR unchanged (email=${newEmail}), skipping update.`);
return;
}
configObj.sessions[found.index].cookie = allCookiesString;
const newFileContent = "export const config = " + JSON.stringify(configObj, null, 4);
fs.writeFileSync(CONFIG_FILE_PATH, newFileContent, "utf8");
sysLogger.debug(`Cookie updated (email=${newEmail})`);
} catch (err) {
sysLogger.warn("Error during cookie update:", err);
}
});
}
/**
*
* @param {import('puppeteer-core').Page} page
*/
export function updateLocalConfigCookieByEmailNonBlocking(page) {
// Ensure async
setImmediate(() => {
updateLocalConfigCookieByEmail(page).catch(err =>
sysLogger.error("Cookie update error:", err)
);
});
}
@@ -1,49 +1,50 @@
import crypto from 'crypto';
export function getRandomInt(min, max) {
min = Math.ceil(min);
max = Math.floor(max);
return Math.floor(Math.random() * (max - min + 1)) + min;
}
// 插入乱码
export function insertGarbledText(content) {
const enableGarbledStart = process.env.ENABLE_GARBLED_START === 'true';
const enableGarbledEnd = process.env.ENABLE_GARBLED_END === 'true';
if (!enableGarbledStart && !enableGarbledEnd) {
return content;
}
// 生成指定长度的随机乱码
function generateGarbledText(length) {
return crypto.randomBytes(length).toString('hex');
}
let garbledContent = content;
// 配置参数
const startMinLength = parseInt(process.env.GARBLED_START_MIN_LENGTH) || 1000;
const startMaxLength = parseInt(process.env.GARBLED_START_MAX_LENGTH) || 5000;
const endGarbledLength = parseInt(process.env.GARBLED_END_LENGTH) || 500;
if (enableGarbledStart) {
const startGarbledLength = getRandomInt(startMinLength, startMaxLength);
const byteLength = Math.ceil(startGarbledLength / 2);
// 生成乱码
const startPlaceholder = generateGarbledText(byteLength);
garbledContent = startPlaceholder + '\n\n\n' + garbledContent.trim();
}
if (enableGarbledEnd) {
const byteLength = Math.ceil(endGarbledLength / 2);
const endPlaceholder = generateGarbledText(byteLength);
garbledContent = garbledContent.trim() + '\n\n\n' + endPlaceholder;
}
return garbledContent;
import crypto from 'crypto';
import sysLogger from '../utils/sysLogger.mjs';
export function getRandomInt(min, max) {
min = Math.ceil(min);
max = Math.floor(max);
return Math.floor(Math.random() * (max - min + 1)) + min;
}
// Insert garbage code
export function insertGarbledText(content) {
const enableGarbledStart = process.env.ENABLE_GARBLED_START === 'true';
const enableGarbledEnd = process.env.ENABLE_GARBLED_END === 'true';
if (!enableGarbledStart && !enableGarbledEnd) {
return content;
}
// Generate random garbage of specified length
function generateGarbledText(length) {
return crypto.randomBytes(length).toString('hex');
}
let garbledContent = content;
// Configuration parameters
const startMinLength = parseInt(process.env.GARBLED_START_MIN_LENGTH) || 1000;
const startMaxLength = parseInt(process.env.GARBLED_START_MAX_LENGTH) || 5000;
const endGarbledLength = parseInt(process.env.GARBLED_END_LENGTH) || 500;
if (enableGarbledStart) {
const startGarbledLength = getRandomInt(startMinLength, startMaxLength);
const byteLength = Math.ceil(startGarbledLength / 2);
// Generate garbage
const startPlaceholder = generateGarbledText(byteLength);
garbledContent = startPlaceholder + '\n\n\n' + garbledContent.trim();
}
if (enableGarbledEnd) {
const byteLength = Math.ceil(endGarbledLength / 2);
const endPlaceholder = generateGarbledText(byteLength);
garbledContent = garbledContent.trim() + '\n\n\n' + endPlaceholder;
}
return garbledContent;
}
@@ -1,292 +1,299 @@
import path from "path";
import fs from "fs";
import {fileURLToPath} from 'url';
import {Mutex} from 'async-mutex';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
class Logger {
constructor() {
this.logMutex = new Mutex();
this.logFilePath = path.join(__dirname, 'requests.log');
this.statistics = {};
this.monthStart = this.getMonthStart();
this.today = this.getToday();
this.loadStatistics();
}
getMonthStart() {
const now = new Date();
// 每月第一天
const monthStart = new Date(now.getFullYear(), now.getMonth(), 1);
monthStart.setHours(0, 0, 0, 0);
return monthStart;
}
getToday() {
const now = new Date();
// 获取当天日期
const today = new Date(now.getFullYear(), now.getMonth(), now.getDate());
today.setHours(0, 0, 0, 0);
return today;
}
// 加载日志
loadStatistics() {
this.logMutex.runExclusive(() => {
if (!fs.existsSync(this.logFilePath)) {
fs.writeFileSync(this.logFilePath, '', 'utf8');
return;
}
const data = fs.readFileSync(this.logFilePath, 'utf-8');
const entries = data.split('\n').filter(line => line.trim());
const validEntries = [];
for (const line of entries) {
try {
const logEntry = JSON.parse(line);
// 补全缺少的字段
if (!logEntry.provider) {
logEntry.provider = 'you';
}
if (!logEntry.email) {
logEntry.email = 'unknown';
}
if (!logEntry.mode) {
logEntry.mode = 'default';
}
if (logEntry.model === undefined) {
logEntry.model = 'unknown';
}
if (logEntry.completed === undefined) {
logEntry.completed = false;
}
if (logEntry.unusualQueryVolume === undefined) {
logEntry.unusualQueryVolume = false;
}
// 调整字段顺序
const logEntryArray = [
['provider', logEntry.provider],
['email', logEntry.email],
['time', logEntry.time],
['mode', logEntry.mode],
['model', logEntry.model],
['completed', logEntry.completed],
['unusualQueryVolume', logEntry.unusualQueryVolume],
];
const formattedLogEntry = Object.fromEntries(logEntryArray);
validEntries.push(formattedLogEntry);
} catch (e) {
console.warn(`无法解析的日志,已忽略: ${line}`);
}
}
// 处理有效日志
for (const logEntry of validEntries) {
const logDate = new Date(logEntry.time);
const provider = logEntry.provider;
const email = logEntry.email;
// 初始化 provider
if (!this.statistics[provider]) {
this.statistics[provider] = {};
}
// 初始化邮箱
if (!this.statistics[provider][email]) {
this.statistics[provider][email] = {
allRequests: [], // 所有请求
monthlyRequests: [], // 本月请求
dailyRequests: [], // 当日请求
monthlyStats: {
totalRequests: 0,
defaultModeCount: 0,
customModeCount: 0,
modelCount: {},
},
dailyStats: {
totalRequests: 0,
defaultModeCount: 0,
customModeCount: 0,
modelCount: {},
}
};
}
const stats = this.statistics[provider][email];
stats.allRequests.push(logEntry);
// 本月统计
if (logDate >= this.monthStart) {
stats.monthlyRequests.push(logEntry);
this.updateStatistics(stats.monthlyStats, logEntry);
}
// 当日统计
if (logDate >= this.today) {
stats.dailyRequests.push(logEntry);
this.updateStatistics(stats.dailyStats, logEntry);
}
}
// 对每个 provider 的每个邮箱时间排序
for (const provider in this.statistics) {
for (const email in this.statistics[provider]) {
const stats = this.statistics[provider][email];
stats.allRequests.sort((a, b) => new Date(b.time) - new Date(a.time));
stats.monthlyRequests.sort((a, b) => new Date(b.time) - new Date(a.time));
stats.dailyRequests.sort((a, b) => new Date(b.time) - new Date(a.time));
}
}
// 清理无效数据
const cleanedData = validEntries.map(entry => JSON.stringify(entry)).join('\n') + '\n';
fs.writeFileSync(this.logFilePath, cleanedData);
}).catch(err => {
console.error('loadStatistics() 加锁异常:', err);
});
}
// 更新统计
updateStatistics(stats, logEntry) {
stats.totalRequests++;
if (logEntry.mode === 'default') {
stats.defaultModeCount++;
} else if (logEntry.mode === 'custom') {
stats.customModeCount++;
}
if (logEntry.model) {
if (!stats.modelCount[logEntry.model]) {
stats.modelCount[logEntry.model] = 0;
}
stats.modelCount[logEntry.model]++;
}
}
// 记录请求日志
logRequest({provider, email, time, mode, model, completed, unusualQueryVolume}) {
const logEntryArray = [
['provider', provider || process.env.ACTIVE_PROVIDER || 'you'],
['email', email || 'unknown'],
['time', time],
['mode', mode || 'unknown'],
['model', model || 'unknown'],
['completed', completed || 'unknown'],
['unusualQueryVolume', unusualQueryVolume || 'unknown'],
];
const logEntry = Object.fromEntries(logEntryArray);
// 写日志与更新 statistics
this.logMutex.runExclusive(() => {
fs.appendFileSync(this.logFilePath, JSON.stringify(logEntry) + '\n');
const logDate = new Date(logEntry.time);
const providerName = logEntry.provider;
if (!this.statistics[providerName]) {
this.statistics[providerName] = {};
}
const userEmail = logEntry.email;
if (!this.statistics[providerName][userEmail]) {
this.statistics[providerName][userEmail] = {
allRequests: [],
monthlyRequests: [],
dailyRequests: [],
monthlyStats: {
totalRequests: 0,
defaultModeCount: 0,
customModeCount: 0,
modelCount: {},
},
dailyStats: {
totalRequests: 0,
defaultModeCount: 0,
customModeCount: 0,
modelCount: {},
}
};
}
const stats = this.statistics[providerName][userEmail];
stats.allRequests.push(logEntry);
// 当日统计
if (logDate >= this.today) {
stats.dailyRequests.push(logEntry);
this.updateStatistics(stats.dailyStats, logEntry);
}
// 本月统计
if (logDate >= this.monthStart) {
stats.monthlyRequests.push(logEntry);
this.updateStatistics(stats.monthlyStats, logEntry);
}
}).catch(err => {
console.error('logRequest() 加锁异常:', err);
});
}
// 输出当前统计信息
printStatistics() {
const provider = process.env.ACTIVE_PROVIDER || 'you';
const monthStartStr = this.monthStart.toLocaleDateString('zh-CN', {
year: 'numeric',
month: 'long',
day: 'numeric'
});
const todayStr = this.today.toLocaleDateString('zh-CN', {
year: 'numeric',
month: 'long',
day: 'numeric'
});
if (!this.statistics[provider]) {
console.log(`===== 提供者 ${provider} 没有统计数据 =====`);
return;
}
const emails = Object.keys(this.statistics[provider]).sort();
let hasAnyDailyRequest = false;
console.log(`===== 请求统计信息 (Provider=${provider}) =====`);
for (const email of emails) {
const stats = this.statistics[provider][email];
// 当日是否有请求
if (stats.dailyStats.totalRequests > 0) {
hasAnyDailyRequest = true;
console.log(`用户邮箱: ${email}`);
console.log(`---------- 本月[自 ${monthStartStr} 起] 统计 ----------`);
console.log(`总请求次数: ${stats.monthlyStats.totalRequests}`);
console.log(`default 请求次数: ${stats.monthlyStats.defaultModeCount}`);
console.log(`custom 请求次数: ${stats.monthlyStats.customModeCount}`);
console.log('各模型请求次数:');
for (const [mdl, count] of Object.entries(stats.monthlyStats.modelCount)) {
console.log(` - ${mdl}: ${count}`);
}
console.log(`---------- 今日[${todayStr}]统计 ----------`);
console.log(`总请求次数: ${stats.dailyStats.totalRequests}`);
console.log(`default 请求次数: ${stats.dailyStats.defaultModeCount}`);
console.log(`custom 请求次数: ${stats.dailyStats.customModeCount}`);
console.log('各模型请求次数:');
for (const [mdl, count] of Object.entries(stats.dailyStats.modelCount)) {
console.log(` - ${mdl}: ${count}`);
}
console.log('------------------------------');
}
}
if (!hasAnyDailyRequest) {
console.log(`===== 今日(${todayStr})无任何账号发生请求 =====`);
}
console.log('================================');
}
}
import path from "path";
import fs from "fs";
import {fileURLToPath} from 'url';
import {Mutex} from 'async-mutex';
import sysLogger from '../utils/sysLogger.mjs';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
class Logger {
constructor() {
this.logMutex = new Mutex();
const dataDir = path.join(process.cwd(), 'data');
if (!fs.existsSync(dataDir)) {
fs.mkdirSync(dataDir, { recursive: true });
}
this.logFilePath = path.join(dataDir, 'requests.log');
this.statistics = {};
this.monthStart = this.getMonthStart();
this.today = this.getToday();
this.loadStatistics();
}
getMonthStart() {
const now = new Date();
// First days of month
const monthStart = new Date(now.getFullYear(), now.getMonth(), 1);
monthStart.setHours(0, 0, 0, 0);
return monthStart;
}
getToday() {
const now = new Date();
// Get current day date
const today = new Date(now.getFullYear(), now.getMonth(), now.getDate());
today.setHours(0, 0, 0, 0);
return today;
}
// Load log
async loadStatistics() {
await this.logMutex.runExclusive(async () => {
try {
await fs.promises.access(this.logFilePath, fs.constants.F_OK);
} catch (err) {
await fs.promises.writeFile(this.logFilePath, '', 'utf8');
return;
}
const data = await fs.promises.readFile(this.logFilePath, 'utf-8');
const entries = data.split('\n').filter(line => line.trim());
const validEntries = [];
for (const line of entries) {
try {
const logEntry = JSON.parse(line);
// Fill in missing fields
if (!logEntry.provider) {
logEntry.provider = 'you';
}
if (!logEntry.email) {
logEntry.email = 'unknown';
}
if (!logEntry.mode) {
logEntry.mode = 'default';
}
if (logEntry.model === undefined) {
logEntry.model = 'unknown';
}
if (logEntry.completed === undefined) {
logEntry.completed = false;
}
if (logEntry.unusualQueryVolume === undefined) {
logEntry.unusualQueryVolume = false;
}
// Adjust field order
const logEntryArray = [
['provider', logEntry.provider],
['email', logEntry.email],
['time', logEntry.time],
['mode', logEntry.mode],
['model', logEntry.model],
['completed', logEntry.completed],
['unusualQueryVolume', logEntry.unusualQueryVolume],
];
const formattedLogEntry = Object.fromEntries(logEntryArray);
validEntries.push(formattedLogEntry);
} catch (e) {
sysLogger.warn(`Unparseable log, ignored: ${line}`);
}
}
// Process valid log
for (const logEntry of validEntries) {
const logDate = new Date(logEntry.time);
const provider = logEntry.provider;
const email = logEntry.email;
// Initialize provider
if (!this.statistics[provider]) {
this.statistics[provider] = {};
}
// Initialize email
if (!this.statistics[provider][email]) {
this.statistics[provider][email] = {
allRequests: [], // All requests
monthlyRequests: [], // Requests this month
dailyRequests: [], // Today's requests
monthlyStats: {
totalRequests: 0,
defaultModeCount: 0,
customModeCount: 0,
modelCount: {},
},
dailyStats: {
totalRequests: 0,
defaultModeCount: 0,
customModeCount: 0,
modelCount: {},
}
};
}
const stats = this.statistics[provider][email];
stats.allRequests.push(logEntry);
// This month's statistics
if (logDate >= this.monthStart) {
stats.monthlyRequests.push(logEntry);
this.updateStatistics(stats.monthlyStats, logEntry);
}
// Today's statistics
if (logDate >= this.today) {
stats.dailyRequests.push(logEntry);
this.updateStatistics(stats.dailyStats, logEntry);
}
}
// Sort time for each email of each provider
for (const provider in this.statistics) {
for (const email in this.statistics[provider]) {
const stats = this.statistics[provider][email];
stats.allRequests.sort((a, b) => new Date(b.time) - new Date(a.time));
stats.monthlyRequests.sort((a, b) => new Date(b.time) - new Date(a.time));
stats.dailyRequests.sort((a, b) => new Date(b.time) - new Date(a.time));
}
}
// Clean invalid data
const cleanedData = validEntries.map(entry => JSON.stringify(entry)).join('\n') + '\n';
await fs.promises.writeFile(this.logFilePath, cleanedData);
}).catch(err => {
sysLogger.error('loadStatistics() lock exception:', err);
});
}
// Update statistics
updateStatistics(stats, logEntry) {
stats.totalRequests++;
if (logEntry.mode === 'default') {
stats.defaultModeCount++;
} else if (logEntry.mode === 'custom') {
stats.customModeCount++;
}
if (logEntry.model) {
if (!stats.modelCount[logEntry.model]) {
stats.modelCount[logEntry.model] = 0;
}
stats.modelCount[logEntry.model]++;
}
}
// Record request log
logRequest({provider, email, time, mode, model, completed, unusualQueryVolume}) {
const logEntryArray = [
['provider', provider || 'you'],
['email', email || 'unknown'],
['time', time],
['mode', mode || 'unknown'],
['model', model || 'unknown'],
['completed', completed || 'unknown'],
['unusualQueryVolume', unusualQueryVolume || 'unknown'],
];
const logEntry = Object.fromEntries(logEntryArray);
// Write log and update statistics
this.logMutex.runExclusive(async () => {
await fs.promises.appendFile(this.logFilePath, JSON.stringify(logEntry) + '\n');
const logDate = new Date(logEntry.time);
const providerName = logEntry.provider;
if (!this.statistics[providerName]) {
this.statistics[providerName] = {};
}
const userEmail = logEntry.email;
if (!this.statistics[providerName][userEmail]) {
this.statistics[providerName][userEmail] = {
allRequests: [],
monthlyRequests: [],
dailyRequests: [],
monthlyStats: {
totalRequests: 0,
defaultModeCount: 0,
customModeCount: 0,
modelCount: {},
},
dailyStats: {
totalRequests: 0,
defaultModeCount: 0,
customModeCount: 0,
modelCount: {},
}
};
}
const stats = this.statistics[providerName][userEmail];
stats.allRequests.push(logEntry);
// Today's statistics
if (logDate >= this.today) {
stats.dailyRequests.push(logEntry);
this.updateStatistics(stats.dailyStats, logEntry);
}
// This month's statistics
if (logDate >= this.monthStart) {
stats.monthlyRequests.push(logEntry);
this.updateStatistics(stats.monthlyStats, logEntry);
}
}).catch(err => {
sysLogger.error('logRequest() lock exception:', err);
});
}
// Output current statistics
printStatistics() {
const provider = 'you';
const monthStartStr = this.monthStart.toLocaleDateString('zh-CN', {
year: 'numeric',
month: 'long',
day: 'numeric'
});
const todayStr = this.today.toLocaleDateString('zh-CN', {
year: 'numeric',
month: 'long',
day: 'numeric'
});
if (!this.statistics[provider]) {
sysLogger.info(`===== Provider ${provider} has no statistics =====`);
return;
}
const emails = Object.keys(this.statistics[provider]).sort();
let hasAnyDailyRequest = false;
sysLogger.info(`===== Request Statistics (Provider=${provider}) =====`);
for (const email of emails) {
const stats = this.statistics[provider][email];
// Any requests today
if (stats.dailyStats.totalRequests > 0) {
hasAnyDailyRequest = true;
sysLogger.debug(`User email: ${email}`);
sysLogger.info(`---------- This Month [Since ${monthStartStr} ] Statistics ----------`);
sysLogger.info(`Total requests: ${stats.monthlyStats.totalRequests}`);
sysLogger.info(`Default requests: ${stats.monthlyStats.defaultModeCount}`);
sysLogger.info(`Custom requests: ${stats.monthlyStats.customModeCount}`);
sysLogger.info('Requests by model:');
for (const [mdl, count] of Object.entries(stats.monthlyStats.modelCount)) {
sysLogger.debug(` - ${mdl}: ${count}`);
}
sysLogger.info(`---------- Today [${todayStr}] Statistics ----------`);
sysLogger.info(`Total requests: ${stats.dailyStats.totalRequests}`);
sysLogger.info(`Default requests: ${stats.dailyStats.defaultModeCount}`);
sysLogger.info(`Custom requests: ${stats.dailyStats.customModeCount}`);
sysLogger.info('Requests by model:');
for (const [mdl, count] of Object.entries(stats.dailyStats.modelCount)) {
sysLogger.debug(` - ${mdl}: ${count}`);
}
sysLogger.debug('------------------------------');
}
}
if (!hasAnyDailyRequest) {
sysLogger.info(`===== No account requests today (${todayStr}) =====`);
}
sysLogger.debug('================================');
}
}
export default Logger;
File diff suppressed because it is too large Load Diff
+119 -119
View File
@@ -1,119 +1,119 @@
@echo off
REM 安装依赖包
call npm install
REM 设置代理的网站:youperplexityhappyapi
set ACTIVE_PROVIDER=you
REM 设置指定浏览器,可以是 'chrome', 'edge' 'auto'
set BROWSER_TYPE=auto
REM 设置是否启用手动登录
set USE_MANUAL_LOGIN=true
REM 设置是否隐藏浏览器 (设置浏览器实例较大时,建议设置为true) (只有在`USE_MANUAL_LOGIN=false`时才有效)
set HEADLESS_BROWSER=true
REM 设置启动浏览器实例数量(非并发场景下,建议设置1)
set BROWSER_INSTANCE_COUNT=1
REM 设置会话自动释放时间(单位:秒) (0=禁用自动释放)
set SESSION_LOCK_TIMEOUT=180
REM 设置是否启用并发限制
set ENABLE_DETECTION=true
REM 设置是否启用自动Cookie更新 (USE_MANUAL_LOGIN=false时有效)
set ENABLE_AUTO_COOKIE_UPDATE=false
REM 是否跳过账户验证 (启用时,`ALLOW_NON_PRO`设置无效,可用于账号量多情况)
set SKIP_ACCOUNT_VALIDATION=false
REM 开启请求次数上限(默认限制3次请求) (用于免费账户)
set ENABLE_REQUEST_LIMIT=false
REM 是否允许非Pro账户
set ALLOW_NON_PRO=false
REM 设置自定义终止符(用于处理输出停不下来情况,留空则不启用,使用双引号包裹)
set CUSTOM_END_MARKER="<CHAR_turn>"
REM 设置是否启用延迟发送请求,如果设置false卡发送请求尝试打开它
set ENABLE_DELAY_LOGIC=false
REM 设置是否启用隧道访问
set ENABLE_TUNNEL=false
REM 设置隧道类型 (localtunnel ngrok)
set TUNNEL_TYPE=ngrok
REM 设置localtunnel子域名(留空则为随机域名)
set SUBDOMAIN=
REM 设置 ngrok AUTH TOKEN
REM 这是 ngrok 账户的身份验证令牌。可以在 ngrok 仪表板的 "Auth" 部分找到它。
REM 免费账户和付费账户都需要设置此项。
REM ngrok网站: https://dashboard.ngrok.com
set NGROK_AUTH_TOKEN=
REM 设置 ngrok 自定义域名
REM 这允许使用自己的域名而不是 ngrok 的随机子域名。
REM 注意:此功能仅适用于 ngrok 付费账户。
REM 使用此功能前,请确保已在 ngrok 仪表板中添加并验证了该域名。
REM 格式示例:your-custom-domain.com
REM 如果使用免费账户或不想使用自定义域名,请将此项留空。
set NGROK_CUSTOM_DOMAIN=
REM 设置 https_proxy 代理,可以使用本地的socks5http(s)代理
REM 例如,使用 HTTP 代理:export https_proxy=http://127.0.0.1:7890
REM 或者使用 SOCKS5 代理:export https_proxy=socks5://host:port:username:password
set https_proxy=
REM 设置 PASSWORD API密码
set PASSWORD=
REM 设置 PORT 端口
set PORT=8080
REM 设置AI模型(Claude系列模型直接在酒馆中选择即可使用,修改`AI_MODEL`环境变量可以切换Claude以外的模型,支持的模型名字如下 (请参考官网获取最新模型))
set AI_MODEL=
REM 自定义会话模式
set USE_CUSTOM_MODE=false
REM 启用模式轮换
REM 只有当 USE_CUSTOM_MODEENABLE_MODE_ROTATION 都设置为 true 时,才会启用模式轮换功能。
REM 可以在自定义模式和默认模式之间动态切换
set ENABLE_MODE_ROTATION=false
REM 是否启用隐身模式
set INCOGNITO_MODE=false
REM 设置伪造真role (如果启用,必须使用txt格式上传)
set USE_BACKSPACE_PREFIX=false
REM 设置上传文件格式 (docx txt) gpt_4o 使用txt可能更好破限
set UPLOAD_FILE_FORMAT=txt
REM 设置是否启用 CLEWD 后处理
set CLEWD_ENABLED=false
REM ---------------------------------------------------
REM 控制是否在开头插入乱码
set ENABLE_GARBLED_START=false
REM 设置开头插入乱码最小长度
set GARBLED_START_MIN_LENGTH=1000
REM 设置开头插入乱码最大长度
set GARBLED_START_MAX_LENGTH=5000
REM 设置结尾插入乱码固定长度
set GARBLED_END_LENGTH=500
REM 控制是否在结尾插入乱码
set ENABLE_GARBLED_END=false
REM ---------------------------------------------------
REM 运行 Node.js 应用程序
node index.mjs
REM 暂停脚本执行,等待用户按任意键退出
pause
@echo off
REM Install dependencies
call npm install
REM Set the website of the proxy: you, perplexity, happyapi
set ACTIVE_PROVIDER=you
REM Set the browser type, can be 'chrome', 'edge', or 'auto'
set BROWSER_TYPE=auto
REM Set whether to enable manual login
set USE_MANUAL_LOGIN=true
REM Set whether to hide the browser (recommended to set to true when the browser instance is large) (only effective when `USE_MANUAL_LOGIN=false`)
set HEADLESS_BROWSER=true
REM Set the number of browser instances to start (recommended to set to 1 in non-concurrent scenarios)
set BROWSER_INSTANCE_COUNT=1
REM Set the session auto-release time (in seconds) (0 = disable auto-release)
set SESSION_LOCK_TIMEOUT=180
REM Set whether to enable detection
set ENABLE_DETECTION=true
REM Set whether to enable automatic cookie updates (effective only when `USE_MANUAL_LOGIN=false`)
set ENABLE_AUTO_COOKIE_UPDATE=false
REM Set whether to skip account verification (when enabled, `ALLOW_NON_PRO` is ignored, can be used in scenarios with multiple accounts)
set SKIP_ACCOUNT_VALIDATION=false
REM Set whether to enable request limits (default limit is 3 requests) (for free accounts)
set ENABLE_REQUEST_LIMIT=false
REM Set whether to allow non-Pro accounts
set ALLOW_NON_PRO=false
REM Set the custom end marker (used to handle situations where the output does not stop, leave blank to not use, use double quotes to enclose)
set CUSTOM_END_MARKER="<CHAR_turn>"
REM Set whether to enable delayed sending of requests, if set to false, it will attempt to open the request
set ENABLE_DELAY_LOGIC=false
REM Set whether to enable tunnel access
set ENABLE_TUNNEL=false
REM Set the tunnel type (localtunnel or ngrok)
set TUNNEL_TYPE=ngrok
REM Set the localtunnel subdomain (leave blank for a random domain)
set SUBDOMAIN=
REM Set the ngrok authentication token
REM This is the authentication token for the ngrok account, which can be found in the "Auth" section of the ngrok dashboard.
REM Both free and paid accounts require this to be set.
REM ngrok website: https://dashboard.ngrok.com
set NGROK_AUTH_TOKEN=
REM Set the ngrok custom domain
REM This allows you to use your own domain instead of ngrok's random subdomain.
REM Note: This feature is only available for ngrok paid accounts.
REM Before using this feature, make sure you have added and verified the domain in the ngrok dashboard.
REM Example format: your-custom-domain.com
REM If using a free account or not using a custom domain, leave this blank.
set NGROK_CUSTOM_DOMAIN=
REM Set the https_proxy proxy, can use local socks5 or http(s) proxy
REM For example, using an HTTP proxy: export https_proxy=http://127.0.0.1:7890
REM Or using a SOCKS5 proxy: export https_proxy=socks5://host:port:username:password
set https_proxy=
REM Set the PASSWORD API password
set PASSWORD=
REM Set the PORT port
set PORT=8080
REM Set the AI model (Claude series models can be used directly in the tavern, modifying the `AI_MODEL` environment variable can switch to other models, supported model names are as follows (please refer to the official website for the latest models))
set AI_MODEL=
REM Set custom session mode
set USE_CUSTOM_MODE=false
REM Enable mode rotation
REM Mode rotation is only enabled when both `USE_CUSTOM_MODE` and `ENABLE_MODE_ROTATION` are set to true.
REM Can dynamically switch between custom mode and default mode
set ENABLE_MODE_ROTATION=false
REM Set whether to enable incognito mode
set INCOGNITO_MODE=false
REM Set whether to fake the true role (if enabled, must use txt format for upload)
set USE_BACKSPACE_PREFIX=false
REM Set the upload file format (docx or txt) gpt_4o may be better with txt
set UPLOAD_FILE_FORMAT=txt
REM Set whether to enable CLEWD post-processing
set CLEWD_ENABLED=false
REM ---------------------------------------------------
REM Set whether to insert garbled characters at the beginning
set ENABLE_GARBLED_START=false
REM Set the minimum length of garbled characters to insert at the beginning
set GARBLED_START_MIN_LENGTH=1000
REM Set the maximum length of garbled characters to insert at the beginning
set GARBLED_START_MAX_LENGTH=5000
REM Set the fixed length of garbled characters to insert at the end
set GARBLED_END_LENGTH=500
REM Set whether to insert garbled characters at the end
set ENABLE_GARBLED_END=false
REM ---------------------------------------------------
REM Run the Node.js application
node src/index.mjs
REM Pause script execution, waiting for the user to press any key to exit
pause
Regular → Executable
+46 -46
View File
@@ -1,112 +1,112 @@
#!/bin/bash
# 安装依赖包
# Install dependencies
npm install
# 设置代理的网站:youperplexityhappyapi
# Set proxy provider: you, perplexity, happyapi
export ACTIVE_PROVIDER=you
# 设置指定浏览器,可以是 'chrome', 'edge' 'auto'
# Set specified browser, can be 'chrome', 'edge' or 'auto'
export BROWSER_TYPE=auto
# 设置是否启用手动登录
# Set whether to enable manual login
export USE_MANUAL_LOGIN=true
# 设置是否隐藏浏览器 (设置浏览器实例较大时,建议设置为true) (只有在`USE_MANUAL_LOGIN=false`时才有效)
# Set whether to hide browser (recommended to set to true when browser instance count is large) (only effective when `USE_MANUAL_LOGIN=false`)
export HEADLESS_BROWSER=true
# 设置启动浏览器实例数量(非并发场景下,建议设置1)
# Set number of browser instances to launch (recommended to set 1 for non-concurrent scenarios)
export BROWSER_INSTANCE_COUNT=1
# 设置会话自动释放时间(单位:秒) (0=禁用自动释放)
# Set session auto-release time (unit: seconds) (0=disable auto-release)
export SESSION_LOCK_TIMEOUT=180
# 设置是否启用并发限制
# Set whether to enable concurrency limit
export ENABLE_DETECTION=true
# 设置是否启用自动Cookie更新 (USE_MANUAL_LOGIN=false时有效)
# Set whether to enable auto cookie update (effective when USE_MANUAL_LOGIN=false)
export ENABLE_AUTO_COOKIE_UPDATE=false
# 是否跳过账户验证 (启用时,`ALLOW_NON_PRO`设置无效,可用于账号量多情况)
# Whether to skip account validation (when enabled, `ALLOW_NON_PRO` setting is invalid, usable for large account quantities)
export SKIP_ACCOUNT_VALIDATION=false
# 开启请求次数上限(默认限制3次请求) (用于免费账户)
# Enable request count limit (default limit 3 requests) (for free accounts)
export ENABLE_REQUEST_LIMIT=false
# 是否允许非Pro账户
# Whether to allow non-Pro accounts
export ALLOW_NON_PRO=false
# 设置自定义终止符(用于处理输出停不下来情况,留空则不启用,使用双引号包裹)
# Set custom terminator (for handling output that won't stop, leave empty to disable, wrap in double quotes)
export CUSTOM_END_MARKER="<CHAR_turn>"
# 设置是否启用延迟发送请求,如果设置false卡发送请求尝试打开它
# Set whether to enable delayed request sending, if set to false and requests get stuck try enabling it
export ENABLE_DELAY_LOGIC=false
# 设置是否启用隧道访问
# Set whether to enable tunnel access
export ENABLE_TUNNEL=false
# 设置隧道类型 (localtunnel ngrok)
# Set tunnel type (localtunnel or ngrok)
export TUNNEL_TYPE=ngrok
# 设置localtunnel子域名(留空则为随机域名)
# Set localtunnel subdomain (leave empty for random domain)
export SUBDOMAIN=
# 设置 ngrok AUTH TOKEN
# 这是 ngrok 账户的身份验证令牌。可以在 ngrok 仪表板的 "Auth" 部分找到它。
# 免费账户和付费账户都需要设置此项。
# ngrok网站: https://dashboard.ngrok.com
# Set ngrok AUTH TOKEN
# This is the authentication token for your ngrok account. You can find it in the "Auth" section of the ngrok dashboard.
# Both free and paid accounts need to set this.
# ngrok website: https://dashboard.ngrok.com
export NGROK_AUTH_TOKEN=
# 设置 ngrok 自定义域名
# 这允许使用自己的域名而不是 ngrok 的随机子域名。
# 注意:此功能仅适用于 ngrok 付费账户。
# 使用此功能前,请确保已在 ngrok 仪表板中添加并验证了该域名。
# 格式示例:your-custom-domain.com
# 如果使用免费账户或不想使用自定义域名,请将此项留空。
# Set ngrok custom domain
# This allows using your own domain instead of ngrok's random subdomain.
# Note: This feature only works with ngrok paid accounts.
# Before using this feature, ensure you have added and verified the domain in the ngrok dashboard.
# Format example: your-custom-domain.com
# If using a free account or don't want to use a custom domain, leave this empty.
export NGROK_CUSTOM_DOMAIN=
# 设置 https_proxy 代理,可以使用本地的socks5http(s)代理
# 例如,使用 HTTP 代理:export https_proxy=http://127.0.0.1:7890
# 或者使用 SOCKS5 代理:export https_proxy=socks5://host:port:username:password
# Set https_proxy proxy, can use local socks5 or http(s) proxy
# For example, using HTTP proxy: export https_proxy=http://127.0.0.1:7890
# Or using SOCKS5 proxy: export https_proxy=socks5://host:port:username:password
export https_proxy=
# 设置 PASSWORD API密码
# Set PASSWORD API password
export PASSWORD=
# 设置 PORT 端口
# Set PORT port
export PORT=8080
# 设置AI模型(Claude系列模型直接在酒馆中选择即可使用,修改`AI_MODEL`环境变量可以切换Claude以外的模型,支持的模型名字如下 (请参考官网获取最新模型))
# Set AI model (Claude series models can be selected directly in Tavern, modifying `AI_MODEL` environment variable can switch to models other than Claude, supported model names below (please refer to official website for latest models))
export AI_MODEL=
# 自定义会话模式
# Custom session mode
export USE_CUSTOM_MODE=false
# 启用模式轮换
# 只有当 USE_CUSTOM_MODE ENABLE_MODE_ROTATION 都设置为 true 时,才会启用模式轮换功能。
# 可以在自定义模式和默认模式之间动态切换
# Enable mode rotation
# Mode rotation is only enabled when both USE_CUSTOM_MODE and ENABLE_MODE_ROTATION are set to true.
# Can dynamically switch between custom mode and default mode
export ENABLE_MODE_ROTATION=false
# 是否启用隐身模式
# Whether to enable incognito mode
export INCOGNITO_MODE=false
# 设置上传文件格式 (docx txt) gpt_4o 使用txt可能更好破限
# Set upload file format (docx or txt) gpt_4o may work better with txt for bypassing limits
export UPLOAD_FILE_FORMAT=docx
# ---------------------------------------------------
# 控制是否在开头插入乱码
# Control whether to insert garbled text at the beginning
export ENABLE_GARBLED_START=false
# 设置开头插入乱码最小长度
# Set minimum length of garbled text at the beginning
export GARBLED_START_MIN_LENGTH=1000
# 设置开头插入乱码最大长度
# Set maximum length of garbled text at the beginning
export GARBLED_START_MAX_LENGTH=5000
# 设置结尾插入乱码固定长度
# Set fixed length of garbled text at the end
export GARBLED_END_LENGTH=500
# 控制是否在结尾插入乱码
# Control whether to insert garbled text at the end
export ENABLE_GARBLED_END=false
# ---------------------------------------------------
# 运行 Node.js 应用程序
node index.mjs
# Run Node.js application
node src/index.mjs
read -p "Press any key to exit..."
+291 -588
View File
@@ -1,588 +1,291 @@
# 使用指南 / Usage Guide
## 前提条件 / Prerequisites
1. **安装必要的软件:**
- Node.js
- Git
- Python
2. **获得一个 You.com 账户并订阅 Pro 或 Team 计划,登录账户。**
3. **建议全局代理来确保网络连接稳定。**
4. **如果需要,可以在 `start.bat` 文件中设置代理。**
---
## 设置步骤 / Setup Steps
### 方法一:使用 Cookie 登录(默认情况下)
#### 步骤 1:获取 Cookie
1. **打开浏览器,登录 [you.com](https://you.com)。**
2. **按 `F12` 打开开发者工具,找到 "Console"(控制台)选项卡。**
3. **在控制台中输入以下代码并回车,然后复制所有输出内容(Cookie):**
```javascript
console.log(document.cookie);
```
#### 步骤 2:配置项目
1. **下载或克隆本项目代码,解压缩。**
2. **编辑 `config.example.mjs` 文件,将上一步获取的 Cookie 粘贴进去。**
如果有多个 Cookie,按照以下格式添加,然后将文件另存为 `config.mjs`
```javascript
export const config = {
"sessions": [
{
"cookie": `cookie1`
},
{
"cookie": `cookie2`
},
{
"cookie": `cookie3`
}
]
}
```
#### 步骤 3:配置环境变量
1. **打开 `start.bat` 文件,根据需要设置环境变量。**
#### 步骤 4:启动服务
1. **双击运行 `start.bat`。**
2. **等待程序安装依赖并启动服务。**
#### 步骤 5:配置客户端
1. **在 SillyTavern 中选择 **Custom (OpenAI-compatible)**。**
2. **将反向代理地址设置为 `http://127.0.0.1:8080/v1`。**
3. **反代密码需要填写(随便填一个即可,除非在 `start.bat` 中设置了 `PASSWORD`)。**
4. **开始使用。如果失败或没有结果,尝试多次重试。**
---
### 方法二:使用手动登录
#### 步骤 1:配置 `start.bat`
1. **打开 `start.bat` 文件,将 `USE_MANUAL_LOGIN` 设置为 `true`**
```batch
set USE_MANUAL_LOGIN=true
```
2. **保存并关闭 `start.bat` 文件。**
#### 步骤 2:启动服务并手动登录
1. **重命名 `config.example.mjs` 文件,将文件另存为 `config.mjs`。**
2. **双击运行 `start.bat`。**
3. **程序将启动并自动打开浏览器窗口。**
4. **在弹出的浏览器窗口中手动登录的 You.com 账户。**
5. **登录成功后,程序将自动获取的会话信息。**
#### 步骤 3:配置客户端
*同方法一的步骤5*
---
## 可选配置 / Optional Configurations
### 设置代理 / Set Proxy
如果需要设置代理,请在 `start.bat` 中设置 `http_proxy` 和 `https_proxy` 环境变量。例如:
```batch
set http_proxy=http://127.0.0.1:7890
set https_proxy=http://127.0.0.1:7890
```
*(启动浏览器闪退时,移除代理)*
This project uses the local Chrome browser, which will automatically read and use the system proxy settings.
### 设置 AI 模型 / Set AI Model
可以通过设置 `AI_MODEL` 环境变量来切换使用的模型。支持的模型包括(请参考官网获取最新模型):
- `gpt_4o`
- `gpt_4_turbo`
- `gpt_4`
- `claude_3_5_sonnet`
- `claude_3_opus`
- `claude_3_sonnet`
- `claude_3_haiku`
- `claude_2`
- `llama3`
- `gemini_pro`
- `gemini_1_5_pro`
- `databricks_dbrx_instruct`
- `command_r`
- `command_r_plus`
- `zephyr`
例如:
```batch
set AI_MODEL=claude_3_opus
```
### 启用自定义会话模式 / Enable Custom Chat Mode
启用后,可以缩短系统消息长度、禁用联网、减少等待时间,可能有助于突破限制。
```batch
set USE_CUSTOM_MODE=true
```
### 启用模式轮换 / Enable Mode Rotation
只有当 `USE_CUSTOM_MODE` 和 `ENABLE_MODE_ROTATION` 都设置为 `true` 时,才会启用模式轮换功能。
```batch
set ENABLE_MODE_ROTATION=true
```
### 启用隧道访问 / Enable Tunnel Access
如果需要从外网访问本地服务,可以启用隧道访问。支持 `ngrok` 和 `localtunnel`。
**使用 ngrok**
1. **设置隧道类型:**
```batch
set ENABLE_TUNNEL=true
set TUNNEL_TYPE=ngrok
```
2. **设置 ngrok Auth Token(从 ngrok 仪表板获取):**
```batch
set NGROK_AUTH_TOKEN=your_ngrok_auth_token
```
3. **(可选)设置自定义域名(付费账户):**
```batch
set NGROK_CUSTOM_DOMAIN=your_custom_domain
```
**使用 localtunnel**
1. **设置隧道类型:**
```batch
set ENABLE_TUNNEL=true
set TUNNEL_TYPE=localtunnel
```
2. **(可选)设置子域名:**
```batch
set SUBDOMAIN=your_subdomain
```
---
## 注意事项 / Important Notes
- **关于 Cloudflare 人机验证:**
如果在程序运行过程中弹出人机验证提示,请在30秒内完成验证。
- **关于 `ALLOW_NON_PRO` 设定:**
如果设置为 `true`,允许使用非订阅账户,但功能会受限,可能无法正常使用。
```batch
set ALLOW_NON_PRO=true
```
- **关于 `CUSTOM_END_MARKER` 设定:**
当输出无法停止时,可设置自定义终止符,程序检测到该终止符后将自动停止输出。
```batch
set CUSTOM_END_MARKER="<YOUR_END_MARKER>"
```
- **关于 `ENABLE_DELAY_LOGIC` 设定:**
如果请求卡顿,尝试将其设置为 `true`。
```batch
set ENABLE_DELAY_LOGIC=true
```
- **关于上传文件格式:**
可以选择上传文件的格式为 `docx` 或 `txt`。
```batch
set UPLOAD_FILE_FORMAT=docx
```
- **关于403问题(基本只存在于旧版本)**
这个问题基本只存在于旧版本,新版本由于使用了浏览器模拟访问,已经不容易被拦截。
新版本如果弹出人机验证提示,用户只需要在30秒内点击完成CloudFlare的人机验证,并且等待程序继续处理即可。
cloudflare有一个风控分数。这个和你的TLS指纹、浏览器指纹、IP地址声誉等等有关系
我们这个项目一直用的TLS指纹和浏览器指纹就非常可疑(都是自动化库和Node内置TLS),分数直接拉满
相当于已经预先有了30+30分数,剩下就看IP地址声誉(40分)你拿了几分
(具体分数不详,只是举个例子)
那如果你IP确实白,拿了0分,那你总共分数就是60
假设you那边设置了分数高于80的要跳验证码,那现在就没事
如果你IP黑,拿了超过20分,那你就是>80分,你就要跳验证码,结果就是403
然后最近you觉得被薅狠了,或者别的啥原因,把这个分数设置成60以上的就要跳验证码
结果就我IP有点黑,不管怎么搞都过不去了。
但是同样的IP,你用正常的Google Chrome访问,就没问题,因为它的指纹非常干净,所以前面的指纹分数就很低
就算加上IP声誉分他也没到那条线
总之以上是一个简化的版本,CF抗bot还有很多指标、很多策略
---
## 在 Linux 上部署 / Deploy on Linux
可以使用 Docker 进行部署,请参照项目中的 `Dockerfile`。
---
## 常见问题 / FAQ
**Q:** 如何解决 npm 安装依赖失败的问题?
**A:** 请确保的网络连接稳定,必要时使用全局代理。
**Q:** 为什么程序提示 "两种模式均达到请求上限"?
**A:** 这可能是因为频繁请求导致模式被暂时禁用,建议稍等一段时间再尝试。
**Q:** 如何切换模型?
**A:** 编辑 `start.bat` 中的 `AI_MODEL` 环境变量,设置为想使用的模型名称(已经可以在SillyTavern设置了)。
---
## 免责声明 / Disclaimer
本项目仅供学习和研究使用,请遵守相关法律法规,勿用于任何商业或非法用途。
This project is for learning and research purposes only. Please comply with relevant laws and regulations and do not use it for any commercial or illegal purposes.
---
# Usage Guide
## Prerequisites
1. **Install necessary software:**
- Node.js
- Git
- Python
- Visual C++ Build Tools ([Download link](https://visualstudio.microsoft.com/visual-cpp-build-tools/))
2. **Obtain a You.com account and subscribe to Pro or Team plan, then log in.**
3. **Global proxy is recommended to ensure stable network connection.**
4. **If needed, you can set up a proxy in the `start.bat` file.**
---
## Setup Steps
### Method 1: Login using Cookie (Default)
#### Step 1: Obtain Cookie
1. **Open browser and log in to [you.com](https://you.com).**
2. **Press `F12` to open developer tools, find the "Console" tab.**
3. **Enter the following code in the console and press enter, then copy all output content (Cookie):**
```javascript
console.log(document.cookie);
```
#### Step 2: Configure Project
1. **Download or clone this project code, unzip.**
2. **Edit `config.example.mjs` file, paste the Cookie obtained in the previous step.**
If there are multiple Cookies, add them in the following format, then save the file as `config.mjs`:
```javascript
export const config = {
"sessions": [
{
"cookie": `cookie1`
},
{
"cookie": `cookie2`
},
{
"cookie": `cookie3`
}
]
}
```
#### Step 3: Configure Environment Variables
1. **Open `start.bat` file, set environment variables as needed.**
#### Step 4: Start Service
1. **Double-click to run `start.bat`.**
2. **Wait for the program to install dependencies and start the service.**
#### Step 5: Configure Client
1. **In SillyTavern, select **Custom (OpenAI-compatible)**.**
2. **Set the reverse proxy address to `http://127.0.0.1:8080/v1`.**
3. **Reverse proxy password needs to be filled (any value will do, unless `PASSWORD` is set in `start.bat`).**
4. **Start using. If it fails or there's no result, try multiple retries.**
---
### Method 2: Manual Login
#### Step 1: Configure `start.bat`
1. **Open `start.bat` file, set `USE_MANUAL_LOGIN` to `true`:**
```batch
set USE_MANUAL_LOGIN=true
```
2. **Save and close `start.bat` file.**
#### Step 2: Start Service and Manual Login
1. **Double-click to run `start.bat`.**
2. **The program will start and automatically open a browser window.**
3. **Manually log in to your You.com account in the pop-up browser window.**
4. **After successful login, the program will automatically obtain the session information.**
#### Step 3: Configure Client
*Same as Step 5 in Method 1*
---
## Optional Configurations
### Set Proxy
If you need to set a proxy, please set the `http_proxy` and `https_proxy` environment variables in `start.bat`. For example:
```batch
set http_proxy=http://127.0.0.1:7890
set https_proxy=http://127.0.0.1:7890
```
This project uses the local Chrome browser, which will automatically read and use the system proxy settings.
### Set AI Model
You can switch the model used by setting the `AI_MODEL` environment variable. Supported models include (please refer to the official website for the latest models):
- `gpt_4o`
- `gpt_4_turbo`
- `gpt_4`
- `claude_3_5_sonnet`
- `claude_3_opus`
- `claude_3_sonnet`
- `claude_3_haiku`
- `claude_2`
- `llama3`
- `gemini_pro`
- `gemini_1_5_pro`
- `databricks_dbrx_instruct`
- `command_r`
- `command_r_plus`
- `zephyr`
For example:
```batch
set AI_MODEL=claude_3_opus
```
### Enable Custom Chat Mode
When enabled, it can shorten system message length, disable internet connection, reduce waiting time, which may help break through limitations.
```batch
set USE_CUSTOM_MODE=true
```
### Enable Mode Rotation
Mode rotation will only be enabled when both `USE_CUSTOM_MODE` and `ENABLE_MODE_ROTATION` are set to `true`.
```batch
set ENABLE_MODE_ROTATION=true
```
### Enable Tunnel Access
If you need to access the local service from the external network, you can enable tunnel access. Both `ngrok` and `localtunnel` are supported.
**Using ngrok:**
1. **Set tunnel type:**
```batch
set ENABLE_TUNNEL=true
set TUNNEL_TYPE=ngrok
```
2. **Set ngrok Auth Token (obtain from ngrok dashboard):**
```batch
set NGROK_AUTH_TOKEN=your_ngrok_auth_token
```
3. **(Optional) Set custom domain (paid account):**
```batch
set NGROK_CUSTOM_DOMAIN=your_custom_domain
```
**Using localtunnel:**
1. **Set tunnel type:**
```batch
set ENABLE_TUNNEL=true
set TUNNEL_TYPE=localtunnel
```
2. **(Optional) Set subdomain:**
```batch
set SUBDOMAIN=your_subdomain
```
---
## Important Notes
- **About Cloudflare CAPTCHA:**
If a CAPTCHA prompt pops up during program operation, please complete the verification within 30 seconds.
- **About `ALLOW_NON_PRO` setting:**
If set to `true`, it allows the use of non-subscription accounts, but functionality will be limited and may not work properly.
```batch
set ALLOW_NON_PRO=true
```
- **About `CUSTOM_END_MARKER` setting:**
When the output cannot stop, you can set a custom termination marker. The program will automatically stop output after detecting this marker.
```batch
set CUSTOM_END_MARKER="<YOUR_END_MARKER>"
```
- **About `ENABLE_DELAY_LOGIC` setting:**
If requests are stuck, try setting this to `true`.
```batch
set ENABLE_DELAY_LOGIC=true
```
- **About upload file format:**
You can choose to upload files in `docx` or `txt` format.
```batch
set UPLOAD_FILE_FORMAT=docx
```
- **About 403 issue (mainly exists in old versions)**
This issue mainly exists in old versions. The new version is less likely to be blocked as it uses browser simulation for access.
In the new version, if a CAPTCHA prompt pops up, users only need to complete the CloudFlare CAPTCHA within 30 seconds and wait for the program to continue processing.
Cloudflare has a risk control score. This is related to your TLS fingerprint, browser fingerprint, IP address reputation, etc.
The TLS fingerprint and browser fingerprint used in this project have always been very suspicious (all are automation libraries and Node built-in TLS), directly maxing out the score.
It's equivalent to having 30+30 points in advance, and the rest depends on how many points your IP address reputation takes (40 points)
(The specific scores are not detailed, just an example)
So if your IP is indeed white and takes 0 points, your total score is 60.
Suppose You sets that scores higher than 80 require CAPTCHA, then there's no problem now.
If your IP is black and takes more than 20 points, then you're >80 points, you need to do CAPTCHA, resulting in 403.
Then recently You felt it was being abused too much, or for some other reason, set it so that scores above 60 require CAPTCHA.
As a result, my IP is a bit black, and I can't get through no matter what.
But with the same IP, if you access with normal Google Chrome, there's no problem, because its fingerprint is very clean, so the previous fingerprint score is very low.
Even with the IP reputation score added, it doesn't reach that line.
In short, the above is a simplified version, CF has many more indicators and strategies for anti-bot.
---
## Deploy on Linux
You can deploy using Docker, please refer to the `Dockerfile` in the project.
---
## FAQ
**Q:** How to solve the problem of npm failing to install dependencies?
**A:** Please ensure your network connection is stable, use global proxy if necessary.
**Q:** Why does the program prompt "Both modes have reached the request limit"?
**A:** This may be because frequent requests have caused the mode to be temporarily disabled. It is recommended to wait for a while before trying again.
**Q:** How to switch models?
**A:** Edit the `AI_MODEL` environment variable in `start.bat`, set it to the name of the model you want to use (can now be set in SillyTavern).
---
## Disclaimer
This project is for learning and research purposes only. Please comply with relevant laws and regulations and do not use it for any commercial or illegal purposes.
---
# Usage Guide
## Prerequisites
1. **Install necessary software:**
- Node.js
- Git
- Python
- Visual C++ Build Tools ([Download link](https://visualstudio.microsoft.com/visual-cpp-build-tools/))
2. **Obtain a You.com account and subscribe to Pro or Team plan, then log in.**
3. **Global proxy is recommended to ensure stable network connection.**
4. **If needed, you can set up a proxy in the `start.bat` file.**
---
## Setup Steps
### Method 1: Login using Cookie (Default)
#### Step 1: Obtain Cookie
1. **Open browser and log in to [you.com](https://you.com).**
2. **Press `F12` to open developer tools, find the "Console" tab.**
3. **Enter the following code in the console and press enter, then copy all output content (Cookie):**
```javascript
console.log(document.cookie);
```
#### Step 2: Configure Project
1. **Download or clone this project code, unzip.**
2. **Edit `config.example.mjs` file, paste the Cookie obtained in the previous step.**
If there are multiple Cookies, add them in the following format, then save the file as `config.mjs`:
```javascript
export const config = {
"sessions": [
{
"cookie": `cookie1`
},
{
"cookie": `cookie2`
},
{
"cookie": `cookie3`
}
]
}
```
#### Step 3: Configure Environment Variables
1. **Open `start.bat` file, set environment variables as needed.**
#### Step 4: Start Service
1. **Double-click to run `start.bat`.**
2. **Wait for the program to install dependencies and start the service.**
#### Step 5: Configure Client
1. **In SillyTavern, select **Custom (OpenAI-compatible)**.**
2. **Set the reverse proxy address to `http://127.0.0.1:8080/v1`.**
3. **Reverse proxy password needs to be filled (any value will do, unless `PASSWORD` is set in `start.bat`).**
4. **Start using. If it fails or there's no result, try multiple retries.**
---
### Method 2: Manual Login
#### Step 1: Configure `start.bat`
1. **Open `start.bat` file, set `USE_MANUAL_LOGIN` to `true`:**
```batch
set USE_MANUAL_LOGIN=true
```
2. **Save and close `start.bat` file.**
#### Step 2: Start Service and Manual Login
1. **Double-click to run `start.bat`.**
2. **The program will start and automatically open a browser window.**
3. **Manually log in to your You.com account in the pop-up browser window.**
4. **After successful login, the program will automatically obtain the session information.**
#### Step 3: Configure Client
*Same as Step 5 in Method 1*
---
## Optional Configurations
### Set Proxy
If you need to set a proxy, please set the `http_proxy` and `https_proxy` environment variables in `start.bat`. For example:
```batch
set http_proxy=http://127.0.0.1:7890
set https_proxy=http://127.0.0.1:7890
```
This project uses the local Chrome browser, which will automatically read and use the system proxy settings.
*(If the browser crashes on startup, remove the proxy.)*
### Set AI Model
You can switch the model used by setting the `AI_MODEL` environment variable. Supported models include (please refer to the official website for the latest models):
- `gpt_4o`
- `gpt_4_turbo`
- `gpt_4`
- `claude_3_5_sonnet`
- `claude_3_opus`
- `claude_3_sonnet`
- `claude_3_haiku`
- `claude_2`
- `llama3`
- `gemini_pro`
- `gemini_1_5_pro`
- `databricks_dbrx_instruct`
- `command_r`
- `command_r_plus`
- `zephyr`
For example:
```batch
set AI_MODEL=claude_3_opus
```
### Enable Custom Chat Mode
When enabled, it can shorten system message length, disable internet connection, reduce waiting time, which may help break through limitations.
```batch
set USE_CUSTOM_MODE=true
```
### Enable Mode Rotation
Mode rotation will only be enabled when both `USE_CUSTOM_MODE` and `ENABLE_MODE_ROTATION` are set to `true`.
```batch
set ENABLE_MODE_ROTATION=true
```
### Enable Tunnel Access
If you need to access the local service from the external network, you can enable tunnel access. Both `ngrok` and `localtunnel` are supported.
**Using ngrok:**
1. **Set tunnel type:**
```batch
set ENABLE_TUNNEL=true
set TUNNEL_TYPE=ngrok
```
2. **Set ngrok Auth Token (obtain from ngrok dashboard):**
```batch
set NGROK_AUTH_TOKEN=your_ngrok_auth_token
```
3. **(Optional) Set custom domain (paid account):**
```batch
set NGROK_CUSTOM_DOMAIN=your_custom_domain
```
**Using localtunnel:**
1. **Set tunnel type:**
```batch
set ENABLE_TUNNEL=true
set TUNNEL_TYPE=localtunnel
```
2. **(Optional) Set subdomain:**
```batch
set SUBDOMAIN=your_subdomain
```
---
## Important Notes
- **About Cloudflare CAPTCHA:**
If a CAPTCHA prompt pops up during program operation, please complete the verification within 30 seconds.
- **About `ALLOW_NON_PRO` setting:**
If set to `true`, it allows the use of non-subscription accounts, but functionality will be limited and may not work properly.
```batch
set ALLOW_NON_PRO=true
```
- **About `CUSTOM_END_MARKER` setting:**
When the output cannot stop, you can set a custom termination marker. The program will automatically stop output after detecting this marker.
```batch
set CUSTOM_END_MARKER="<YOUR_END_MARKER>"
```
- **About `ENABLE_DELAY_LOGIC` setting:**
If requests are stuck, try setting this to `true`.
```batch
set ENABLE_DELAY_LOGIC=true
```
- **About upload file format:**
You can choose to upload files in `docx` or `txt` format.
```batch
set UPLOAD_FILE_FORMAT=docx
```
- **About 403 issue (mainly exists in old versions)**
This issue mainly exists in old versions. The new version is less likely to be blocked as it uses browser simulation for access.
In the new version, if a CAPTCHA prompt pops up, users only need to complete the CloudFlare CAPTCHA within 30 seconds and wait for the program to continue processing.
Cloudflare has a risk control score. This is related to your TLS fingerprint, browser fingerprint, IP address reputation, etc.
The TLS fingerprint and browser fingerprint used in this project have always been very suspicious (all are automation libraries and Node built-in TLS), directly maxing out the score.
It's equivalent to having 30+30 points in advance, and the rest depends on how many points your IP address reputation takes (40 points)
(The specific scores are not detailed, just an example)
So if your IP is indeed white and takes 0 points, your total score is 60.
Suppose You sets that scores higher than 80 require CAPTCHA, then there's no problem now.
If your IP is black and takes more than 20 points, then you're >80 points, you need to do CAPTCHA, resulting in 403.
Then recently You felt it was being abused too much, or for some other reason, set it so that scores above 60 require CAPTCHA.
As a result, my IP is a bit black, and I can't get through no matter what.
But with the same IP, if you access with normal Google Chrome, there's no problem, because its fingerprint is very clean, so the previous fingerprint score is very low.
Even with the IP reputation score added, it doesn't reach that line.
In short, the above is a simplified version, CF has many more indicators and strategies for anti-bot.
---
## Deploy on Linux
You can deploy using Docker, please refer to the `Dockerfile` in the project.
---
## FAQ
**Q:** How to solve the problem of npm failing to install dependencies?
**A:** Please ensure your network connection is stable, use global proxy if necessary.
**Q:** Why does the program prompt "Both modes have reached the request limit"?
**A:** This may be because frequent requests have caused the mode to be temporarily disabled. It is recommended to wait for a while before trying again.
**Q:** How to switch models?
**A:** Edit the `AI_MODEL` environment variable in `start.bat`, set it to the name of the model you want to use (can now be set in SillyTavern).
---
## Disclaimer
This project is for learning and research purposes only. Please comply with relevant laws and regulations and do not use it for any commercial or illegal purposes.
File diff suppressed because it is too large Load Diff