mirrors/jsproxy
1
0
Fork 0
mirror of https://github.com/EtherDream/jsproxy synced 2024-11-21 09:36:51 +00:00
Code Issues Packages Projects Releases Wiki Activity

new version

Browse Source
This commit is contained in:
zjcqoo 2019-04-23 12:14:27 +08:00
parent 731af7b60a
commit 977fce511e
56 changed files with 437 additions and 3097 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

210
README.md
View File

@ -1,212 +1,108 @@
# 在线预览
https://www.gk.jsproxy.tk
https://zjcqoo.github.io/-----https://www.google.com
（目前仍在更新中，最好使用隐身模式访问，避免缓存导致的问题）
# 安装部署
## 依赖
* OpenResty
* acme.sh
* node.js / webpack / webpack-cli
CentOS7 可执行 `./server/setup.sh` 一键安装。
[之前版本](https://github.com/EtherDream/jsproxy/tree/first-ver)已不再更新，但[演示服务](https://jsproxy.tk/)仍保留一段时间。
## 配置
# 安装
首先需要一个域名，例如 example.com，解析 @ 和 * 到服务器 IP。
在项目的根目录下新建 `dnsconf` 文件：
新建一个名为 `jsproxy` 用户，在其主目录安装 nginx：
```bash
DOMAIN=example.com
DNS_ID=dns_xx
export xx_id=xxx
export xx_key=xxxxxx
useradd jsproxy -g nobody
su jsproxy
cd ~
git clone --depth=1 git@github.com:EtherDream/jsproxy.git server
cd server
./setup-nginx.sh
```
第一个为域名，后面三个参考 [acme.sh dns api](https://github.com/Neilpang/acme.sh/tree/master/dnsapi)。
安装过程若有依赖缺失，可尝试（CentOS 为例）：
执行 `./build.sh`。该过程会申请 SSL 证书，时间可能较长。
执行 `./server/run.sh` 开启服务。
访问 `https://example.com` 即可进入首页。
> 本项目使用了 `brotli_static` 指令，如果当前的 nginx 不支持，可在 `server/nginx.conf` 配置中将其注释，或参考 `server/setup.sh` 重新编译 nginx。
## 扩展
编辑 `sitelist.txt` 文件，可配置站点别名，格式为 `别名 主机名`。配置完成后需要执行 `build.sh` 更新。
执行 `./server/run.sh reload` 重启服务。（该命令的参数和 nginx -s 意义一样，当然也可以自己管理 nginx 服务）
访问 `https://别名.example.com` 即可进入相应站点。
由于 HTTPS 证书不支持多级通配，所以别名数量是有限的（好像 acme.sh 只支持 30 几个）
对于普通的域名，例如 `www.host.com` 则转换成 `www-dot-host-dot-com.example.com` 的格式，即 `.` 变成 `-dot-`。（原本就有 `-dot-` 字符的域名暂未考虑）
# 功能特点
## 性能开销
本代理主要功能都运行在客户端，最大程度减少服务端计算量。前端通过 `Service Worker` 拦截和处理资源，同时注入一个 JS 到页面顶部，实现一些辅助功能。
服务端则非常简单，直接利用 nginx 反向代理功能，并且不修改内容（只修改 HTTP 头），避免处理内容的开销，以及原始数据解压再压缩的开销（或者不压缩时流量开销）。
例如现在流行的 br 压缩，压缩比高但压缩成本很大。因此让代理服务器只转发而不操作数据，可节省大量资源。
## 域名模型
本代理将不同的目标站点作为独立的子域名，例如：
```text
so.jsproxy.tk => stackoverflow.com
gk.jsproxy.tk => www.google.com.hk
```bash
yum install -y \
gcc gcc-c++ \
pcre pcre-devel \
openssl openssl-devel \
zlib zlib-devel
```
这在一定程度上隔离了站点之间的数据，例如 Cookie、Storage 等。
## 测试
该模型支持目标站点子域和主域 Cookie 共存：
启动服务：
![](docs/sub-root-cookie.png)
```bash
./server/run.sh
```
另外页面中的辅助脚本，也会对部分 DOM API 进行重写，模拟一个沙盒环境。
访问：https://etherdream.github.io/jsproxy-localtest/-----https://github.com/
例如脚本设置 Cookie 时，会触发钩子程序对赋值进行调整：
![](https://raw.githubusercontent.com/EtherDream/jsproxy-localtest/temp/preview.png)
![](docs/js-set-cookie.png)
注意，**当前项目只提供接口服务**，浏览器端脚本和页面不在本项目。这样做是为了让接口和界面分离，意义参见后续。
类似的还有：
![](docs/domain-model.png)
# 部署
使得代理对页面尽可能保持透明。
参考 `gen-cert` 目录，为自己的域名申请证书，然后修改 `nginx.conf` 中域名相关的配置（默认被注释），以及 DNS 地址（默认是 114.114.114.114）。
浏览器端项目位于：https://github.com/EtherDream/jsproxy-browser
## 路径修正
参考备注，修改服务器域名，之后将 www 目录发布到 Web 空间即可。
前端脚本会对资源、超链接、表单、弹窗的 URL 进行修正。
（目前还不完善，之后将实现动态配置，无需修改 JS 代码）
后端代理会对请求头的 `Referer`、`Origin` 字段进行修正，减少被拦截的可能。
![](docs/login1.png)
# 安全策略
![](docs/login2.png)
如果不希望代理访问内网，可执行 `setup-ipset.sh`，避免 SSRF 风险。
目前测试了 GitHub、Twitter 可以登陆，Google 登陆还有一些问题。
该脚本可禁止 `jsporxy` 用户访问内网（针对 TCP）。nginx 之外的程序也生效，但不影响其他用户。
当然请不要在测试服务器里输入隐私数据。
# 服务管理
# 存在问题
重启服务：`./run.sh reload`
该代理目前仍存在较多问题，主要有：
关闭服务：`./run.sh quit`
## 普通域名模式没有子域
参数和 nginx -s 相同。
由于 `www-dot-host-dot-com.example.com` 并非 `host-dot-com.example.com` 的子域，因此这种模式下 cookie 和 domain 都无法支持域模型。
未来可能会尝试把所有站点都放在同个域名下，例如 `https://example.com/host.com/path/to`，这样就无需考虑域名的问题。当然这种方案需要重写更多的 API 以确保数据隔离，甚至还要自己维护 cookie 的携带，难度比较大。
# CHANGELOG
## v0.0.1
## location hook
虽然目前仍为概念演示状态，但相比最初版本，有了很大变化：
由于 `window` 和 `document` 对象的 `location` 属性无法重写，导致很多网站的脚本在读写路径时会出问题。
* 不再使用二级域名
目前在代码层解决这个问题：通过 Service Worker 以及 API 钩子拦截 JS 代码，然后将其中的 `location` 字符串替换成 `__location`，从而将操作转到我们的对象上。
由于二级域名的方案缺陷太多，例如 HTTPS 证书问题，DNS 性能和安全问题等，最终不再使用二级域名，而是只用单个域名，目标 URL 放在路径里。例如：
由于这种方式简单粗暴，有时会把正则、字符串、属性名的 location 也替换了，导致代码出现问题。因此之后会尝试在 AST 层面进行调整，当然缺点是比较耗时，尤其对于很大的 JS。
https://zjcqoo.github.io/-----https://www.google.com
当然，如果 `location` 不是字面出现的，比如 `obj[key]` 形式，那么这种方案仍不可行，除非调整 `window` 和 `document`。但它们也可以不通过字面获取，例如通过 `this` 也可以获取 `window`，更别提 `eval` 等等。。。所以网站本身若真想访问 `location`，我们还是很难阻止的。
当然这也会产生很多新问题，例如无法支持 Cookie 等、页面之前没有同源策略限制等。
因此这里给 Web 开发者一个建议：如果想检测当前页面 URL 是否为钓鱼网站，最好不要出现字面量的 `window`、`location` 获取 URL，而是通过动态的方式进行获取，以防落入上述这种低级的陷阱。
对于 Cookie 问题，目前通过 JS 来维护，而不用浏览器原生的（当然还有不少细节没实现）。这样的好处是前后端可以分离，前端的页面可以放在 CDN、GitHub Pages 上，我们的服务器只提供代理接口。
这样一个页面可使用多个服务器，实现线路实时切换、负载均衡等效果。
## 多进程问题
由于 Service Worker 无法拦截第三方站点的框架页，因此会出现 iframe 逃脱代理的情况。
* 服务端优化
目前尝试对框架元素的 `src` 属性进行拦截，同时监控 DOM 创建事件，将新增的框架调整成我们的 URL。当然这里面还涉及到 `about:`、`blob:`、`data:` 等协议，会有些麻烦，暂未实现。
安全改进：由于 Web 页面托管在第三方站点上，自己的服务器无需开启 443 端口，因此也无需 root 运行。同时支持 IP 黑名单功能，防止 SSRF 攻击。
另外新创建的 `Worker`、`SharedWorker` 暂时也没有注入辅助 JS 代码，还在调研中。
代码改进：接口代理使用固定的 URL（`/http` 和 `/ws`），不再使用任意路径，代码干净了很多。
至于业务方的 `ServiceWorker`，目前是直接拒绝其使用的，因为这会和代理本身的 `ServiceWorker` 冲突。以后再调研两者是否能较好的共存。
* 提供一个首页
## 很多地方需要优化
由于目前还只是个概念验证的状态，很多代码都是临时写的，之后稳定了再重构和完善。
另外测试案例也没有，估计有一大堆 BUG 还没发现。
# 优化探索
YY 一些优化方案，以后有时间探索。
## 流量的优先级
因为我们是在前端拦截流量，所以能了解每个请求的具体用途，从而可更好的设置优先级。例如在流量压力较大时，优先满足网页、脚本等流量，推迟视频、动画等流量，确保主要功能不受影响。
## 脚本离线分析
由于前端修改 JS 比较耗性能，因此可事先把各大网站的常用脚本在本地分析，然后上传到 nginx 缓存里。这样浏览器就不需要实时计算了，可以大幅降低开销。并且离线分析可以更加深入，对于动态访问 `location` 的情况也能覆盖到，甚至完全不局限于修改 `location` 的功能，而是更通用的调整，例如去广告，增加其他功能等等。
另外对于常用的内联脚本，也可将离线分析结果进行下发，浏览器运行时只需简单查表，避免大量在线计算。
## 资源本地加速
进一步，还考虑可以把常用网站的静态资源预先下回本地，部署到附近的 CDN 上，或者 [打包成图片上传到各大免费图床、相册里](https://yq.aliyun.com/articles/236582)，提供给 Service Worker 更快的访问通道。这样可大幅加快网站访问速度，并且节省代理服务器的流量！
## 缓存重新压缩
虽然大部分网站都开启了 HTTP 传输压缩，并且不少支持 br 格式，但考虑到压缩成本，很多网站并没有将压缩率调到最大。而我们的代理服务器，显然也不会为了节省那么一点流量，牺牲大量 CPU 去做解压和压缩。
但是，这个过程可以离线去做，尤其对于那些 CPU 过剩而流量紧缺的服务器。我们可将空闲时的 CPU 资源用于 nginx cache 最高级 br 压缩。甚至还可以对非 CORS 请求的图片进行更高程度的压缩，并且转换成 WebP 格式，进一步降低流量开销。
## 动态数据压缩
有些网页内容很大却关闭了缓存，例如 google 首页，每次访问都要重新下载一次，浪费不少流量。但是让代理服务器强制缓存也是不行的，因为页面里可能包含了用户信息，缓存的话就会串号导致隐私问题。
然而这些页面的绝大部分都是相同的，每次重复传输实属不必。因此，我们可预先分析出那些不涉及隐私的公共子串，将其部署在本地 CDN 上。代理在返回数据时，重复部分用索引代替，从而可减少传输流量，提高访问速度。更进一步，甚至可以尝试把网页反推回模板，这样只需传输模板变量就可以！
对于那些流量接收免费、发送计费的服务器来说，这是个值得考虑的优化方案。
# 初衷
春节期间由于家里电脑上不了 google 很是不爽，平时用惯了公司自带的科学上网，好久没维护自己的都不能用了，于是一气之下写了这个程序。
其实很久以前也尝试过类似的，但都十分简陋。这次决定做个完善的，充分用上浏览器的新技术和黑魔法，顺便再熟悉下 nginx 的技术细节。
当然制作过程并不顺利，遇到各种问题。因此先实现了一个简单的 google 代理，之后的问题就可以通过它解决了。于是用「开发中的代理」搜索「代理开发中」遇到的问题，然后不断改进。或许这就叫自举😂
尽管折腾了整个春节，但毕竟不是寒假才几天时间，所以仍是个半成品。不过用来浏览常见的编程网站是没问题的，甚至还能刷推看视频。
当然要做到完善还需不少时间，暂时先分享个半成品吧~
# 后续
之后还会将它用于以下技术的研究：
* 网站镜像 / 沙盒化
* 钓鱼网站攻防检测
* 资源访问端上加速
当然请勿将本项目用于访问非法用途，否则后果自负。
虽然依旧简陋，但比之前好。
# License

4
allowed-sites.txt Normal file
View File

@ -0,0 +1,4 @@
# no pathname
http://localhost 1;
https://etherdream.github.io 1;
https://zjcqoo.github.io 1;

161
api.conf Normal file
View File

@ -0,0 +1,161 @@
if ($_origin_allowed = '') {
return 404 'ERROR: origin `$http_origin` is not allowed';
}
if ($http_x_jsproxy) {
return 404 'ERROR: circular dependency';
}
proxy_set_header x-jsproxy 1;
proxy_set_header Connection $http_connection;
set $_url '';
set $_ver '';
location = /preflight {
internal;
more_set_headers
'access-control-allow-origin: *'
'access-control-allow-methods: GET,POST,PUT,DELETE,HEAD,OPTIONS'
'access-control-allow-headers: --url,--referer,--cookie,--origin,--ext,--aceh,--ver,accept,accept-charset,accept-encoding,accept-language,accept-datetime,authorization,cache-control,content-length,content-type,date,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,max-forwards,pragma,range,te,upgrade,upgrade-insecure-requests,x-requested-with,chrome-proxy'
'access-control-max-age: 1728000'
;
return 204;
}
# HTTP(S) Proxy
location = /http {
if ($request_method = 'OPTIONS') {
rewrite ^ /preflight;
}
# decode req headers
access_by_lua_block {
local hdrs, err = ngx.req.get_headers()
local extHdrs
for k, v in pairs(hdrs) do
if k:sub(1, 2) ~= '--' then
goto continue
end
ngx.req.clear_header(k)
k = k:sub(3)
if k == 'url' then
ngx.var._url = v
elseif k == 'ver' then
ngx.var._ver = v
elseif k == 'aceh' then
ngx.ctx._aceh = 1
elseif k == 'ext' then
extHdrs = require('cjson').decode(v)
else
ngx.req.set_header(k, v)
end
::continue::
end
if extHdrs then
for k, v in pairs(extHdrs) do
ngx.req.set_header(k, v)
end
end
}
proxy_cache my_cache;
proxy_pass $_url;
more_set_headers
'server: $upstream_http_server'
'content-security-policy'
'content-security-policy-report-only'
'x-frame-options'
;
# encode res headers
header_filter_by_lua_block {
local expose = '*'
local detail = (ngx.ctx._aceh == 1)
local vary = '--url'
local h, err = ngx.resp.get_headers()
for k, v in pairs(h) do
if
-- headers to escape --
k == 'access-control-allow-origin' or
k == 'access-control-expose-headers' or
k == 'location' or
k == 'set-cookie'
then
if type(v) == 'table' then
for i = 1, #v do
local x = i .. '-' .. k
ngx.header[x] = v[i]
if detail then
expose = expose .. ',' .. x
end
end
else
local x = '--' .. k
ngx.header[x] = v
if detail then
expose = expose .. ',' .. x
end
end
ngx.header[k] = nil
elseif k == 'vary' then
if type(v) == 'table' then
vary = vary .. ', ' .. table.concat(v, ', ')
else
vary = vary .. ', ' .. v
end
elseif detail and
-- not simple header --
k ~= 'cache-control' and
k ~= 'cache-language' and
k ~= 'content-type' and
k ~= 'expires' and
k ~= 'last-modified' and
k ~= 'pragma'
then
expose = expose .. ',' .. k
end
end
if detail then
expose = expose .. ',--s'
ngx.header['--t'] = '1'
end
ngx.header['access-control-expose-headers'] = expose
ngx.header['access-control-allow-origin'] = '*'
ngx.header['vary'] = vary
ngx.header['--s'] = ngx.status
ngx.status = 200
}
}
# WebSocket Proxy
location = /ws {
access_by_lua_block {
local query, err = ngx.req.get_uri_args()
for k, v in pairs(query) do
if k == 'url__' then
ngx.var._url = v
elseif k == 'ver__' then
ngx.var._ver = v
else
ngx.req.set_header(k, v)
end
end
}
proxy_pass $_url;
}

27
browser/home/index.html
View File

@ -1,27 +0,0 @@
<!doctype html>
<html>
<head>
<title>Page Sandbox Demo</title>
<meta charset="utf-8">
<base target="_blank">
<style>
#txtURL {
width: 300px;
}
</style>
</head>
<body>
<h1>网页沙盒</h1>
<div>
URL:
<input id="txtURL" value="https://www.google.com.hk">
<button id="btnGo">Go</button>
</div>
<script src="x.js"></script>
<script>
btnGo.onclick = function() {
open(txtURL.value)
}
</script>
</body>
</html>

3
browser/proxy/debug.sh
View File

@ -1,3 +0,0 @@
JS=../../server/www/x.js
rm -f $JS.br
webpack -w -o $JS --mode development

15
browser/proxy/package.json
View File

@ -1,15 +0,0 @@
{
"name": "jsproxy-client",
"version": "0.0.1",
"description": "",
"main": "boot.js",
"directories": {
"lib": "lib"
},
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "EtherDream",
"license": "MIT"
}

2
browser/proxy/release.sh
View File

@ -1,2 +0,0 @@
webpack --mode production
brotli -f -o ../../server/www/x.js.br dist/main.js

158
browser/proxy/src/fakeloc.js
View File

@ -1,158 +0,0 @@
import * as urlx from "./urlx";
/**
* @param {string} url
*/
function decOrigin(url) {
const u = new URL(url)
urlx.decUrlObj(u)
return u.origin
}
function setup(obj, fakeLoc) {
Reflect.defineProperty(obj, '__location', {
get() {
return fakeLoc
},
set(val) {
console.log('[jsproxy] %s set location: %s', obj, val)
fakeLoc.href = val
}
})
}
/**
* @param {Window} win
*/
export function init(win) {
let loc = win.location
// TODO: iframe 场合下存在问题
// 比如 youtube 首页缺少这个判断会报错
if (loc.href === 'about:blank') {
loc = win.top.location
}
const fakeLoc = Object.setPrototypeOf({
get href() {
// console.log('[jsproxy] get location.href')
return urlx.decUrlStr(loc.href)
},
get protocol() {
// TODO: 未考虑非 https 的页面 URL
return loc.protocol
},
get host() {
// TODO: 未考虑带端口的页面 URL
// console.log('[jsproxy] get location.host')
return urlx.decHost(loc.host)
},
get hostname() {
// console.log('[jsproxy] get location.hostname')
return urlx.decHost(loc.hostname)
},
get port() {
// TODO: 未考虑带端口的页面 URL
return loc.port
},
get pathname() {
return loc.pathname
},
get search() {
return loc.search
},
get hash() {
return loc.hash
},
get origin() {
// console.log('[jsproxy] get location.origin')
return decOrigin(loc.origin)
},
get ancestorOrigins() {
// TODO: DOMStringList[]
// console.log('[jsproxy] get location.ancestorOrigins')
return [...loc.ancestorOrigins].map(decOrigin)
},
set href(val) {
console.log('[jsproxy] set location.href:', val)
loc.href = urlx.encUrlStr(val, loc)
},
set protocol(val) {
const u = new URL(loc)
// TODO:
},
set host(val) {
console.log('[jsproxy] set location.host:', val)
// TODO:
},
set hostname(val) {
console.log('[jsproxy] set location.hostname:', val)
loc.hostname = urlx.encHost(val)
},
set port(val) {
console.log('[jsproxy] set location.port:', val)
// TODO:
},
set pathname(val) {
loc.pathname = val
},
set search(val) {
loc.search = val
},
set hash(val) {
loc.hash = val
},
reload() {
loc.reload(...arguments)
},
replace(val) {
if (val) {
console.log('[jsproxy] location.replace:', val)
arguments[0] = urlx.encUrlStr(val, loc)
}
loc.replace(...arguments)
},
assign(val) {
if (val) {
console.log('[jsproxy] location.assign:', val)
arguments[0] = urlx.encUrlStr(val, loc)
}
loc.assign(...arguments)
},
toString() {
const val = loc.toString(...arguments)
return urlx.decUrlStr(val)
},
toLocaleString() {
const val = loc.toLocaleString(...arguments)
return urlx.decUrlStr(val)
},
}, loc.constructor.prototype)
setup(win, fakeLoc)
setup(win.document, fakeLoc)
}

247
browser/proxy/src/hook.js
View File

@ -1,247 +0,0 @@
export const RETURN = {}
const {
apply,
getOwnPropertyDescriptor,
defineProperty,
} = Reflect
const rawMap = new WeakMap()
/**
* @param {Window} win
*/
export function createHook(win) {
/**
* hook function
*
* @param {object} obj
* @param {string} key
* @param {Function} factory
*/
function func(obj, key, factory) {
const oldFn = obj[key]
if (!oldFn) {
return false
}
const newFn = factory(oldFn)
for (const k in oldFn) {
newFn[k] = oldFn[k]
}
newFn.prototype = oldFn.prototype
rawMap.set(newFn, oldFn)
obj[key] = newFn
return true
}
/**
* hook property
*
* @param {object} obj
* @param {string} key
* @param {Function} g
* @param {Function} s
*/
function prop(obj, key, g, s) {
const desc = getOwnPropertyDescriptor(obj, key)
if (!desc) {
return false
}
if (g) {
func(desc, 'get', g)
}
if (s) {
func(desc, 'set', s)
}
defineProperty(obj, key, desc)
return true
}
function hookElemProp(proto, name, onget, onset) {
prop(proto, name,
getter => function() {
const val = getter.call(this)
return onget.call(this, val)
},
setter => function(val) {
val = onset.call(this, val)
if (val == RETURN) {
return
}
setter.call(this, val)
}
)
}
const toLCase = ''.toLocaleLowerCase
const elemProto = win.Element.prototype
const rawGetAttr = elemProto.getAttribute
const rawSetAttr = elemProto.setAttribute
const tagAttrHandlersMap = {}
const tagTextHandlerMap = {}
const tagKeySetMap = {}
const tagKeyGetMap = {}
function attr(tag, proto, ...handlers) {
let hasBind, hasAttr
let keySetMap, keyGetMap
handlers.forEach(v => {
// 带划线的 attr 属性名，转换成驼峰形式的 prop 属性名。
// 例如 `http-equiv` -> `httpEquiv`
const prop = v.name.replace(/-(\w)/g,
(_, char) => char.toUpperCase()
)
hookElemProp(proto, prop, v.onget, v.onset)
// #text
if (prop === 'innerText') {
tagTextHandlerMap[tag] = v
return
}
// attribute
if (tagAttrHandlersMap[tag]) {
tagAttrHandlersMap[tag].push(v)
hasBind = true
} else {
tagAttrHandlersMap[tag] = [v]
tagKeySetMap[tag] = {}
tagKeyGetMap[tag] = {}
}
if (!keySetMap) {
keySetMap = tagKeySetMap[tag]
keyGetMap = tagKeyGetMap[tag]
}
const key = toLCase.call(v.name)
keySetMap[key] = v.onset
keyGetMap[key] = v.onget
hasAttr = true
})
if (hasBind || !hasAttr) {
return
}
// 如果之前调用过 setAttribute，直接返回上次设置的值；
// 如果没有调用过，则返回 onget 的回调值。
func(proto, 'getAttribute', oldFn => function(name) {
const key = toLCase.call(name)
const onget = keyGetMap[key]
if (!onget) {
return apply(oldFn, this, arguments)
}
const lastVal = this['_k' + key]
if (lastVal !== undefined) {
return lastVal
}
const val = apply(oldFn, this, arguments)
return onget(val)
})
func(proto, 'setAttribute', oldFn => function(name, val) {
const key = toLCase.call(name)
const onset = keySetMap[key]
if (onset) {
this['_k' + key] = val
const ret = onset.call(this, val)
if (ret === RETURN) {
return
}
arguments[1] = ret
}
return apply(oldFn, this, arguments)
})
// TODO: setAttributeNode
// ...
}
/**
* @param {Text} node
* @param {object} handler
* @param {Element} elem
*/
function parseNewTextNode(node, handler, elem) {
const val = node.nodeValue
const ret = handler.onset.call(elem, val)
if (ret === RETURN) {
return
}
node.nodeValue = ret
}
/**
* @param {Element} elem
* @param {object} handler
*/
function parseNewElemNode(elem, handler) {
const name = handler.name
if (!elem.hasAttribute(name)) {
return
}
const val = rawGetAttr.call(elem, name)
const ret = handler.onset.call(elem, val)
if (ret === RETURN) {
return
}
rawSetAttr.call(elem, name, ret)
}
/**
* @param {MutationRecord[]} mutations
*/
function parseMutations(mutations) {
mutations.forEach(mutation => {
mutation.addedNodes.forEach(node => {
switch (node.nodeType) {
case 1: // ELEMENT_NODE
const handlers = tagAttrHandlersMap[node.tagName]
handlers && handlers.forEach(v => {
parseNewElemNode(node, v)
})
break
case 3: // TEXT_NODE
const elem = node.parentElement
if (elem) {
const handler = tagTextHandlerMap[elem.tagName]
if (handler) {
parseNewTextNode(node, handler, elem)
}
}
break
}
})
})
}
const observer = new win.MutationObserver(parseMutations)
observer.observe(win.document, {
childList: true,
subtree: true,
})
// win.addEventListener('DOMContentLoaded', e => {
// parseMutations(observer.takeRecords())
// observer.disconnect()
// })
// hide source code
func(win.Function.prototype, 'toString', oldFn => function() {
return apply(oldFn, rawMap.get(this) || this, arguments)
})
return {
func,
prop,
attr,
}
}

37
browser/proxy/src/hostlist.js
View File

@ -1,37 +0,0 @@
// THIS FILE WAS GENERATED BY build.sh
// DO NOT MODIFY
export const MY_ROOT = 'jsproxy.tk'
export const HOST_LIST = [
['gg', 'google.com'],
['gc', 'google.cn'],
['gk', 'google.com.hk'],
['gu', 'googleusercontent.com'],
['gs', 'googlesource.com'],
['wk', 'wikipedia.org'],
['m.wk', 'm.wikipedia.org'],
['so', 'stackoverflow.com'],
['se', 'stackexchange.com'],
['sf', 'serverfault.com'],
['su', 'superuser.com'],
['au', 'askubuntu.com'],
['gh', 'github.com'],
['qr', 'quora.com'],
['ux', 'unix.com'],
['mz', 'mozilla.org'],
['w3', 'w3schools.com'],
['cr', 'chromium.org'],
['my', 'myspace.com'],
['fb', 'facebook.com'],
['yt', 'youtube.com'],
['tw', 'twitter.com'],
['fl', 'flickr.com'],
['rd', 'reddit.com'],
['bg', 'blogger.com'],
['wp', 'wordpress.com'],
['md', 'medium.com'],
['hn', 'hackernoon.com'],
['yh', 'yahoo.com'],
['bc', 'bbc.com'],
['th', 'twitch.tv'],
['sc', 'steamcommunity.com'],
]

13
browser/proxy/src/index.js
View File

@ -1,13 +0,0 @@
function main() {
if ('onclick' in self) {
// page env
return require('./page.js')
}
if ('onfetch' in self) {
// sw env
return require('./sw.js')
}
return require('./worker.js')
}
main()

77
browser/proxy/src/inject.js
View File

@ -1,77 +0,0 @@
import * as urlx from "./urlx";
import * as util from './util.js'
import * as jsfilter from './jsfilter.js'
const RES_HOST = urlx.getMyRootHost()
const HELPER_URL = `//${RES_HOST}/x.js`
// 为了简化注入位置的分析，这里直接插到 HTML 开头
// 所以页面里会出现两个 <!DOCTYPE>
const HTML_BEG = util.strToBytes(
`<!DOCTYPE html><script src="${HELPER_URL}"></script>`
)
// Worker
const WORKER_BEG = util.strToBytes(
`importScripts('${HELPER_URL}');`
)
/**
* @param {Response} res
* @param {Object} resOpt
*/
export function htmlRemote(res, resOpt) {
const reader = res.body.getReader()
let injected
const stream = new ReadableStream({
async pull(controller) {
if (!injected) {
injected = true
controller.enqueue(HTML_BEG)
}
const r = await reader.read()
if (r.done) {
controller.close()
return
}
controller.enqueue(r.value)
}
})
return new Response(stream, resOpt)
}
// 处理 data、blob 协议的页面
export function htmlLocal(uri) {
// TODO:
}
/**
* @param {Response} res
* @param {Object} resOpt
*/
export async function jsRemote(res, resOpt, charset) {
// 之后会分析语法树，所以不使用流模式
const buf = await res.arrayBuffer()
const ret = await jsfilter.parseBin(buf, charset)
if (ret) {
resOpt.headers = new Headers(resOpt.headers)
resOpt.headers.set('content-type', 'text/javascript')
}
return new Response(ret || buf, resOpt)
}
export function workerRemote(res, resOpt, charset) {
// TODO:
}
// 处理 data、blob 协议的 Worker
export function workerLocal(data) {
// TODO:
}

31
browser/proxy/src/jsfilter.js
View File

@ -1,31 +0,0 @@
import * as util from './util.js'
/**
* @param {string} code
*/
export function parseSync(code) {
// TODO: parse js ast
let match
code = code.replace(/(\b)location(\b)/g, (s, $1, $2) => {
match = true
return $1 + '__location' + $2
})
if (match) {
return code
}
}
/**
* @param {Uint8Array} buf
*/
export async function parseBin(buf, charset) {
const str = util.bytesToStr(buf, charset)
const ret = parseSync(str)
if (ret) {
return util.strToBytes(ret)
}
if (!util.isUtf8(charset)) {
return util.strToBytes(str)
}
}

171
browser/proxy/src/nav.js
View File

@ -1,171 +0,0 @@
import * as urlx from './urlx.js';
/**
* page navigate intercept
*
* @param {Window} win
* @param {Hook} hook
*/
export function init(win, hook) {
const {
location,
Reflect,
} = win
const {
apply,
} = Reflect
const linkProto = win.HTMLAnchorElement.prototype
const areaProto = win.HTMLAreaElement.prototype
const formProto = win.HTMLFormElement.prototype
function hookNavAttr(tag, proto, name) {
hook.attr(tag, proto, {
name,
onget(val) {
const u = new URL(val, location)
urlx.unpack(u)
return u.href
},
onset(val) {
const u = new URL(val, location)
urlx.pack(u, false, false)
return u.href
}
})
}
hookNavAttr('A', linkProto, 'href')
hookNavAttr('AREA', areaProto, 'href')
hookNavAttr('FORM', formProto, 'action')
// TODO:
function hookLinkProp(proto) {
hook.prop(proto, 'hostname',
getter => function() {
const val = getter.call(this)
return val
},
setter => function(val) {
console.log('[jsproxy] set link hostname:', val)
setter.call(this, val)
}
)
hook.prop(proto, 'host',
getter => function() {
const val = getter.call(this)
return val
},
setter => function(val) {
console.log('[jsproxy] set link host:', val)
setter.call(this, val)
}
)
hook.prop(proto, 'protocol',
getter => function() {
const val = getter.call(this)
return val
},
setter => function(val) {
console.log('[jsproxy] set link protocol:', val)
setter.call(this, val)
}
)
hook.prop(proto, 'port',
getter => function() {
const val = getter.call(this)
return val
},
setter => function(val) {
console.log('[jsproxy] set link port:', val)
setter.call(this, val)
}
)
hook.prop(proto, 'search',
getter => function() {
const val = getter.call(this)
return val
},
setter => function(val) {
console.log('[jsproxy] set link search:', val)
setter.call(this, val)
}
)
}
hookLinkProp(linkProto)
hookLinkProp(areaProto)
/**
* @param {HTMLAnchorElement | HTMLAreaElement | HTMLFormElement} el
* @param {string} prop
*/
function processElem(el, prop) {
const urlStr = el[prop]
if (urlStr) {
el[prop] = urlStr
}
}
function linkClickHook(oldFn) {
return function() {
processElem(this, 'href')
return apply(oldFn, this, arguments)
}
}
hook.func(linkProto, 'click', linkClickHook)
hook.func(areaProto, 'click', linkClickHook)
hook.func(formProto, 'submit', oldFn => function() {
processElem(this, 'action')
return apply(oldFn, this, arguments)
})
// hook window.open()
hook.func(win, 'open', oldFn => function(url) {
if (url) {
const u = new URL(url, location)
urlx.pack(u, false, false)
arguments[0] = u.href
}
return apply(oldFn, this, arguments)
})
//
// hook <base>
//
const baseProto = win.HTMLBaseElement.prototype
hook.attr('BASE', baseProto, {
name: 'href',
onget(val) {
return urlx.decUrlStr(val)
},
onset(val) {
// console.log('[jsproxy] set base.href:', val)
// val = getFinalUrl(val)
return urlx.encUrlStr(val, location)
}
})
//
// hook <meta>
//
const metaProto = win.HTMLMetaElement.prototype
hook.attr('META', metaProto, {
name: 'http-equiv',
onget(val) {
// TODO:
return val
},
onset(val) {
return val
}
})
}

393
browser/proxy/src/page.js
View File

@ -1,393 +0,0 @@
import {createHook} from './hook.js'
import * as urlx from './urlx.js'
import * as util from './util.js'
import * as nav from './nav.js'
import * as jsfilter from './jsfilter.js'
import * as fakeloc from './fakeloc.js'
/**
* @param {Window} win
*/
function initWin(win) {
if (!win) {
return
}
try {
if (win.Math.__flag) {
return // setuped
}
win.Math.__flag = 1
} catch (err) {
return // not same origin
}
const {
// WeakSet,
// Reflect,
// RegExp,
// URL,
// Proxy,
document,
location,
navigator,
} = win
const {
apply,
construct,
} = Reflect
const isExtPageMode = urlx.isMyExtHost(location.hostname)
// hook Function
// hook.func(window, 'Function', oldFn => function() {
// return apply(oldFn, this, arguments)
// })
const hook = createHook(win)
nav.init(win, hook)
// hook window/document.location
fakeloc.init(win, hook)
// hook document.domain
const docProto = win.Document.prototype
hook.prop(docProto, 'domain',
getter => function() {
const val = getter.call(this)
return urlx.decHost(val)
},
setter => function(val) {
if (isExtPageMode) {
console.warn('[jsproxy] unsafe domain')
val = urlx.getMyExtHost()
} else {
val = urlx.encHost(val)
}
setter.call(this, val)
}
)
// hook document.cookie
const R_COOKIE_DOMAIN = /(?<=;\s*domain=)[^;]+/i
hook.prop(docProto, 'cookie', null,
setter => function(val) {
val = val.replace(R_COOKIE_DOMAIN, rHost => {
if (isExtPageMode) {
return ''
}
if (rHost[0] === '.') {
rHost = rHost.substr(1)
}
const vHost = urlx.encHost(rHost)
if (urlx.isMyRootHost(vHost)) {
console.warn('[jsproxy] invalid cookie domain:', rHost, vHost)
}
return vHost
})
setter.call(this, val)
}
)
// uri api
function getUriHook(getter) {
return function() {
const val = getter.call(this)
return urlx.decUrlStr(val)
}
}
hook.prop(docProto, 'referrer', getUriHook)
hook.prop(docProto, 'URL', getUriHook)
hook.prop(docProto, 'documentURI', getUriHook)
hook.prop(win.Node.prototype, 'baseURI', getUriHook)
// disable ServiceWorker
const swProto = win.ServiceWorkerContainer.prototype
if (swProto) {
hook.func(swProto, 'register', oldFn => function() {
console.warn('access serviceWorker.register blocked')
return new Promise(function() {})
})
hook.func(swProto, 'getRegistration', oldFn => function() {
console.warn('access serviceWorker.getRegistration blocked')
return new Promise(function() {})
})
hook.func(swProto, 'getRegistrations', oldFn => function() {
console.warn('access serviceWorker.getRegistrations blocked')
return new Promise(function() {})
})
}
//
// hook history
//
function historyStateHook(oldFn) {
return function(_0, _1, url) {
if (url) {
arguments[2] = urlx.encUrlStr(url, location)
}
// console.log('[jsproxy] history.replaceState', url)
return apply(oldFn, this, arguments)
}
}
const historyProto = win.History.prototype
hook.func(historyProto, 'pushState', historyStateHook)
hook.func(historyProto, 'replaceState', historyStateHook)
//
hook.func(navigator, 'registerProtocolHandler', oldFn => function(_0, url, _1) {
console.log('registerProtocolHandler:', arguments)
return apply(oldFn, this, arguments)
})
// hook Performance API
hook.prop(win.PerformanceEntry.prototype, 'name', getUriHook)
//
// hook iframe
//
const iframeProto = win.HTMLIFrameElement.prototype
hook.prop(iframeProto, 'contentWindow',
getter => function() {
const win = getter.call(this)
initWin(win)
return win
}
)
hook.prop(iframeProto, 'contentDocument',
getter => function() {
const doc = getter.call(this)
if (doc) {
initWin(doc.defaultView)
}
return doc
}
)
hook.attr('IFRAME', iframeProto, {
name: 'src',
onget(val) {
return urlx.decUrlStr(val)
},
onset(val) {
val = urlx.encUrlStr(val, location)
console.log('[jsproxy] set <iframe> src', val)
return val
}
})
const embedProto = win.HTMLEmbedElement.prototype
hook.attr('EMBED', embedProto, {
name: 'src',
onget(val) {
console.log('[jsproxy] get <embed> src:', val)
return val
},
onset(val) {
console.log('[jsproxy] set <embed> src:', val)
return val
}
})
const objectProto = win.HTMLObjectElement.prototype
hook.attr('OBJECT', objectProto, {
name: 'data',
onget(val) {
console.log('[jsproxy] get <object> src:', val)
return val
},
onset(val) {
console.log('[jsproxy] set <object> src:', val)
return val
}
})
const frames = win.frames
win.frames = new Proxy(frames, {
get(_, key) {
if (typeof key === 'number') {
console.log('get frames index:', key)
const win = frames[key]
initWin(win)
return win
} else {
return frames[key]
}
}
})
//
// hook message origin
//
hook.func(win, 'postMessage', oldFn => function(msg, origin) {
// origin 必须是完整的 URL（不接受 // 开头的相对协议）
if (origin && origin !== '*') {
arguments[1] = urlx.encUrlStr(origin)
}
return apply(oldFn, this, arguments)
})
hook.prop(win.MessageEvent.prototype, 'origin', getUriHook)
//
// hook xhr
//
const xhrProto = win.XMLHttpRequest.prototype
hook.func(xhrProto, 'open', oldFn => function(_0, url, async) {
if (url) {
arguments[1] = urlx.encUrlStr(url, location)
}
if (async === false) {
console.log('[jsproxy] sync xhr is disabled')
arguments[2] = true
}
return apply(oldFn, this, arguments)
})
hook.func(win, 'fetch', oldFn => function(v) {
if (v && v.url) {
// v is Request
url = urlx.encUrlStr(url)
arguments[0] = new Request(url, v)
} else {
// v is string
arguments[0] = urlx.encUrlStr(v, location)
}
return apply(oldFn, this, arguments)
})
// hook Worker
function workHook(oldFn) {
return function(url) {
if (url) {
console.log('[jsproxy] new worker:', url)
arguments[0] = urlx.encUrlStr(url, location)
}
return construct(oldFn, arguments)
}
}
hook.func(win, 'Worker', workHook)
hook.func(win, 'SharedWorker', workHook)
// hook WebSocket
hook.func(win, 'WebSocket', oldFn => function(url) {
if (url) {
const u = new URL(url)
urlx.pack(u, true, true)
arguments[0] = u.href
}
return construct(oldFn, arguments)
})
const scriptProto = win.HTMLScriptElement.prototype
hook.attr('SCRIPT', scriptProto,
// 强制使用 utf-8 编码，方便 SW 编码
{
name: 'charset',
onget(val) {
return this._charset || val
},
onset(val) {
if (!util.isUtf8(val)) {
val = 'utf-8'
}
this._charset = val
return val
}
},
// 禁止设置内容校验
{
name: 'integrity',
onget(val) {
return this._integrity
},
onset(val) {
this._integrity = val
return ''
}
},
{
name: 'innerText',
onget(val) {
return val
},
onset(val) {
updateScript(this)
return val
}
})
// text 属性只有 prop 没有 attr
let scriptTextSetter
function scriptGetJs(getter) {
return function() {
return getter.call(this)
}
}
function scriptSetJs(setter) {
scriptTextSetter = setter
return function(val) {
updateScript(this)
setter.call(this, val)
}
}
hook.prop(scriptProto, 'text', scriptGetJs, scriptSetJs)
const JS_MIME = {
'': true,
'text/javascript': true,
'application/javascript': true,
'module': true,
}
/**
* @param {HTMLScriptElement} elem
*/
function updateScript(elem) {
const type = elem.type
if (!JS_MIME[type]) {
return
}
const code = elem.text
if (!code) {
return
}
if (elem.__parsed) {
return
}
const ret = jsfilter.parseSync(code)
if (ret) {
scriptTextSetter.call(elem, ret)
}
elem.__parsed = true
}
}
initWin(self)
if (self !== parent) {
parent.postMessage('__READY', '*')
}
document.currentScript.remove()
console.log('[jsproxy] helper inited', location.href)

153
browser/proxy/src/sw.js
View File

@ -1,153 +0,0 @@
import * as urlx from './urlx.js'
import * as util from './util.js'
import * as inject from './inject.js'
const TYPE_HTML = 1
const TYPE_JS = 2
const TYPE_WORKER = 2
/**
*
* @param {Request} req
* @param {URL} urlObj
*/
async function forward(req, urlObj, redirNum = 0) {
const hasCors = (req.mode === 'cors')
urlx.pack(urlObj, true, hasCors)
let reqType = 0
if (req.mode === 'navigate') {
reqType = TYPE_HTML
} else {
const dest = req.destination
if (dest === 'script') {
reqType = TYPE_JS
} else if (dest === 'worker') {
reqType = TYPE_WORKER
}
}
const reqOpt = {
// mode: reqType ? 'cors' : req.mode,
mode: 'cors',
method: req.method,
headers: req.headers,
credentials: req.credentials,
signal: req.signal,
// referrerPolicy: 'no-referrer',
referrer: req.referrer,
}
if (req.method === 'POST') {
// TODO: 解决 stream is lock 的错误
const buf = await req.arrayBuffer()
if (buf.byteLength > 0) {
reqOpt.body = buf
}
}
const res = await fetch(urlObj, reqOpt)
const resStatus = res.status
// https://fetch.spec.whatwg.org/#statuses
const isEmpty =
(resStatus === 101) ||
(resStatus === 204) ||
(resStatus === 205) ||
(resStatus === 304)
if (isEmpty) {
return res
}
const resHdr = res.headers
const resOpt = {
status: resStatus,
statusText: res.statusText,
headers: resHdr,
}
// fake redirect
const isRedir =
(resStatus === 311) ||
(resStatus === 312) ||
(resStatus === 317) ||
(resStatus === 318)
if (isRedir) {
const newUrl = resHdr.get('location')
if (newUrl) {
// 重定向到相对路径，是基于请求的 URL 计算（不是页面的 URL）
const u = new URL(newUrl, urlObj)
if (req.redirect === 'follow') {
if (redirNum > 5) {
return new Response('TOO_MUCH_REDIR')
}
return forward(req, u, redirNum + 1)
}
urlx.encUrlObj(u)
// urlx.delFlag(u)
resOpt.headers = new Headers(resHdr)
resOpt.headers.set('location', u)
}
resOpt.status = resStatus - 10
return new Response(res.body, resOpt)
}
if (reqType === 0) {
return res
}
// content-type: text/html; ...; charset="gbk"
const ctVal = resHdr.get('content-type') || ''
const [, mime, charset] = ctVal
.toLocaleLowerCase()
.match(/([^;]*)(?:.*?charset=['"]?([^'"]+))?/)
// if (charset && !util.isUtf8(charset)) {
// console.warn('[jsproxy] charset:', charset, urlObj.href)
// }
if (reqType === TYPE_HTML) {
if (mime === 'text/html') {
return inject.htmlRemote(res, resOpt)
}
} else if (reqType === TYPE_JS) {
return inject.jsRemote(res, resOpt, charset)
}
return res
}
async function proxy(e, urlObj) {
// TODO: 读取本地缓存的资源，以及从本地 CDN 加速
try {
return await forward(e.request, urlObj)
} catch (err) {
console.warn('[jsproxy] forward err:', err)
}
}
self.onfetch = function(e) {
const u = new URL(e.request.url)
// internal resource (helper.js)
if (urlx.isMyRootHost(u.host)) {
return
}
if (urlx.isHttpProto(u.protocol)) {
e.respondWith(proxy(e, u))
} else {
console.log('ignore non-http res:', u.href)
}
}
self.onactivate = function() {
clients.claim()
}
console.log('[jsproxy] sw inited')

326
browser/proxy/src/urlx.js
View File

@ -1,326 +0,0 @@
import {MY_ROOT, HOST_LIST} from './hostlist.js'
const MY_ROOT_DOT = '.' + MY_ROOT
const MY_EXT = 'ext' + MY_ROOT_DOT
const MY_EXT_DOT = '.' + MY_EXT
const HOST_ENC_MAP = {}
const HOST_DEC_MAP = {}
HOST_LIST.forEach(([alias, rHost]) => {
HOST_ENC_MAP[rHost] = alias
HOST_DEC_MAP[alias] = rHost
})
export function getMyRootHost() {
return MY_ROOT
}
export function getMyExtHost() {
return MY_EXT
}
function makeReg(tmpl, map, suffix = '') {
const list = Object.keys(map)
.join('|')
.replace(/\./g, '\\.')
const [a, b, c] = tmpl.raw
if (suffix) {
suffix = suffix.replace(/\./g, '\\.') + c
}
return RegExp(a + list + b + suffix)
}
const R_HOST_ENC = makeReg`^([\w-]+\.)??(${HOST_ENC_MAP})$`
const R_HOST_DEC = makeReg`^([\w-]+\.)??(${HOST_DEC_MAP})${MY_ROOT_DOT}$`
/**
* encode host (rHost to vHost)
*
* @param {string} rHost
* @example
* 'twitter.com' -> 'tw.mysite.net'
* 'www.google.com' -> 'www.gg.mysite.net'
* 'www.google.com.hk' -> 'www.gk.mysite.net'
* 'unsupport.com' -> 'unsupport-dot-com.mysite.net'
* 'not-support.com' -> 'not-support-dot-com.mysite.net'
* '*.mysite.net' -> '*.mysite.net'
* 'mysite.net' -> 'mysite.net'
*/
function _encHost(rHost) {
if (isMyHost(rHost)) {
return rHost
}
// 内置域名（替换成短别名）
const m = rHost.match(R_HOST_ENC)
if (m) {
const [, sub, root] = m
const vHost = HOST_ENC_MAP[root]
if (vHost) {
return (sub || '') + vHost + MY_ROOT_DOT
}
}
// 外置域名（将 `.` 替换成 `-dot-`）
if (rHost.includes('-dot-')) {
console.warn('invalid host:', rHost)
return rHost
}
return rHost.replace(/\./g, '-dot-') + MY_EXT_DOT
}
/**
* decode host (vHost to rHost)
*
* @param {string} vHost
* @returns {string}
* return *null* if vHost not ends with `HOST_SUFFIX`
* or not in `HOST_LIST`
*
* @example
* 'gg.mysite.net' -> 'google.com'
* 'www.gg.mysite.net' -> 'www.google.com'
* 'not-support-dot-com.mysite.net' -> 'not-support.com'
* 'www-dot-mysite-dot-net.mysite.net' -> 'www.mysite.net'
* 'www.google.com' -> null
* 'x.mysite.net' -> null
*/
function _decHost(vHost) {
if (isMyExtHost(vHost)) {
return vHost
.slice(0, -MY_EXT_DOT.length)
.replace(/-dot-/g, '.')
}
const m = vHost.match(R_HOST_DEC)
if (m) {
const [, sub, root] = m
const rHost = HOST_DEC_MAP[root]
if (rHost) {
return (sub || '') + rHost
}
}
return null
}
const encCache = {}
const decCache = {}
/**
* @param {string} rHost
*/
export function encHost(rHost) {
let ret = encCache[rHost]
if (!ret) {
ret = _encHost(rHost)
encCache[rHost] = ret
}
return ret
}
export function decHost(vHost) {
let ret = decCache[vHost]
if (!ret) {
ret = _decHost(vHost)
decCache[vHost] = ret
}
return ret
}
/**
* @param {string} host
*/
export function isMyHost(host) {
return isMyRootHost(host) || isMySubHost(host)
}
/**
* @param {string} host
*/
export function isMyRootHost(host) {
return host === MY_ROOT
}
/**
* @param {string} host
*/
export function isMySubHost(host) {
return host.endsWith(MY_ROOT_DOT)
}
/**
* @param {string} host
*/
export function isMyExtHost(host) {
return host.endsWith(MY_EXT_DOT)
}
/**
* @param {string} path
*/
export function isHttpProto(path) {
return /^https?:/.test(path)
}
/**
* encode urlObj.hostname to vHost
*
* @param {URL} urlObj
*/
export function encUrlObj(urlObj) {
urlObj.hostname = encHost(urlObj.hostname)
}
/**
* @param {URL} urlObj
* @returns {boolean}
*/
export function decUrlObj(urlObj) {
const host = decHost(urlObj.hostname)
if (host) {
urlObj.hostname = host
}
return !!host
}
/**
* @param {string} url
* 需编码的 URL 字符串，可以是完整 URL，或相对路径、相对协议。
*
* @param {string | URL} baseUrl
* 如果 url 不完整，需指定一个基地址。
* 如果未指定基地址，并且 url 不完整，则返回 url 本身。
*/
export function encUrlStr(url, baseUrl) {
if (!url) {
return url
}
try {
var urlObj = new URL(url, baseUrl)
} catch (err) {
return url
}
encUrlObj(urlObj)
return urlObj.href
}
/**
* @param {string} url
*/
export function decUrlStr(url) {
if (!url) {
return url
}
try {
var urlObj = new URL(url)
} catch (err) {
return url
}
return decUrlObj(urlObj) ? urlObj.href : url
}
/**
* @param {URL} urlObj
* @param {boolean} hasSw
* @param {boolean} hasCors
*/
export function pack(urlObj, hasSw, hasCors) {
let unsafe = false
switch (urlObj.protocol) {
case 'https:':
break
case 'wss:':
break
case 'http:':
unsafe = true
urlObj.protocol = 'https:'
break
case 'ws:':
unsafe = true
urlObj.protocol = 'wss:'
break
default:
// 例如 chrome-extension:
return
}
encUrlObj(urlObj)
const port = urlObj.port
// 都未设置，则不加 flag
if (!hasSw && !unsafe && !hasCors && !port) {
return
}
if (port && port !== '443') {
urlObj.port = '443'
}
let flag = '' +
(+hasSw) +
(+unsafe) +
(+hasCors) +
port
//
// 使用 urlObj.searchParams 设置参数会对已有参数进行编码，例如：
// new URL('https://s.yimg.com/zz/combo?yui:/3.12.0/yui/yui-min.js')
// 设置参数后 :/ 等字符会被编码，导致资源无法加载。
//
let args = urlObj.search
urlObj.search = args.replace(/&flag__=[^&]*|$/, _ => {
// 出现 ?&flag= 也没事，后端用同样的方法删除该标记
return (args ? '' : '?') + '&flag__=' + flag
})
}
// /**
// * @param {URL} urlObj
// */
// export function delFlag(urlObj) {
// urlObj.search = urlObj.search.replace(/&flag__=[^&]*/, '')
// }
/**
* @param {URL} urlObj
*/
export function unpack(urlObj) {
const flag = urlObj.searchParams.get('flag__')
if (!flag) {
return
}
const unsafe = (flag[1] === '1')
const port = flag.substr(3)
switch (urlObj.protocol) {
case 'https:':
if (unsafe) {
urlObj.protocol = 'http:'
}
break
case 'wss:':
if (unsafe) {
urlObj.protocol = 'ws:'
}
break
default:
console.warn('unpack:', urlObj)
return
}
if (port) {
urlObj.port = port
}
decUrlObj(urlObj)
}

23
browser/proxy/src/util.js
View File

@ -1,23 +0,0 @@
const ENC = new TextEncoder()
/**
* @param {string} str
*/
export function strToBytes(str) {
return ENC.encode(str)
}
/**
* @param {BufferSource} bytes
* @param {string} charset
*/
export function bytesToStr(bytes, charset = 'utf-8') {
return new TextDecoder(charset).decode(bytes)
}
/**
* @param {string} label
*/
export function isUtf8(label) {
return /^utf-?8$/i.test(label)
}

0
browser/proxy/src/worker.js
View File

17
browser/setup/gen.sh
View File

@ -1,17 +0,0 @@
DST=../../server/www/__setup.html
html-minifier \
--collapse-whitespace \
--remove-comments \
--remove-optional-tags \
--remove-redundant-attributes \
--remove-script-type-attributes \
--remove-tag-whitespace \
--use-short-doctype \
--remove-attribute-quotes \
--minify-css true \
--minify-js '{"toplevel": true, "ie8": true}' \
-o $DST \
index.html
brotli -f $DST

38
browser/setup/index.html
View File

@ -1,38 +0,0 @@
<p id="t"></p>
<script>
function reload() {
var curr = Date.now()
try {
var last = +sessionStorage._ts || 0
if (curr - last < 100) {
return setTimeout(reload, 2000)
}
sessionStorage._ts = curr
} catch (err) {
}
location.reload()
}
function onfail(err) {
t.innerHTML = err.message
}
if (top === self) {
t.innerHTML = 'loading...'
}
var sw = navigator.serviceWorker
if (!sw || !self.ReadableStream) {
t.innerHTML = '请使用最新版 Chrome 浏览器访问'
} else {
sw.getRegistration().then(function(reg) {
if (reg) {
reload()
} else {
sw.register('/__sw.js')
.then(reload)
.catch(onfail)
}
})
}
</script>

108
build.sh
View File

@ -1,108 +0,0 @@
source ./dnsconf
svc_port=443
acme_args="-d $DOMAIN -d *.$DOMAIN -d *.ext.$DOMAIN"
js_arr_items=""
ngx_vhost_rhost="\
*.ext.$DOMAIN \$_vhost_dec_ext;
"
ngx_rhost_vhost="\
default \$_rhost_enc_ext.ext.$DOMAIN;
"
while read alias host
do
[ -z "$host" ] && continue
acme_args+=" -d *.$alias.$DOMAIN"
js_arr_items+=" ['$alias', '$host'],
"
# rhost to vhost map
dot_str=${host//[^.]}
dot_num=${#dot_str}
ngx_rhost_vhost+="\
$host $alias.$DOMAIN;
www.$host www.$alias.$DOMAIN;
*.$host \$_rhost_slice_$dot_num.$alias.$DOMAIN;
"
# vhost to rhost map
dot_str=${alias//[^.]}
dot_num=${#dot_str}
ngx_vhost_rhost+="\
$alias.$DOMAIN $host;
www.$alias.$DOMAIN www.$host;
*.$alias.$DOMAIN \$_vhost_slice_$dot_num.$host;
"
done < sitelist.txt
# gen nginx conf
echo "$ngx_vhost_rhost" > ./server/include/vhost-rhost.map
echo "$ngx_rhost_vhost" > ./server/include/rhost-vhost.map
echo "\
server_name $DOMAIN;
listen $svc_port ssl;" > ./server/include/host-root.conf
echo "\
server_name *.$DOMAIN;
listen $svc_port ssl;" > ./server/include/host-wild.conf
echo "\
ssl_certificate cert/${DOMAIN}.fullchain.rsa.cer;
ssl_certificate_key cert/${DOMAIN}.rsa.key;
ssl_certificate cert/${DOMAIN}.fullchain.ecc.cer;
ssl_certificate_key cert/${DOMAIN}.ecc.key;
" > ./server/include/cert.conf
echo "\
return 200 'importScripts(\"//${DOMAIN}/x.js\")';
" > ./server/include/x-js.conf
# gen ssl cert
ACME=~/.acme.sh/acme.sh
$ACME \
--issue \
--dns $DNS_ID \
$acme_args
$ACME \
--issue \
--dns $DNS_ID \
$acme_args \
--keylength ec-256
$ACME \
--install-cert -d $DOMAIN \
--key-file ./server/cert/$DOMAIN.rsa.key \
--fullchain-file ./server/cert/$DOMAIN.fullchain.rsa.cer
$ACME \
--install-cert -d $DOMAIN --ecc \
--key-file ./server/cert/$DOMAIN.ecc.key \
--fullchain-file ./server/cert/$DOMAIN.fullchain.ecc.cer
# gen js file
cd ./browser/proxy
echo "\
// THIS FILE WAS GENERATED BY build.sh
// DO NOT MODIFY
export const MY_ROOT = '$DOMAIN'
export const HOST_LIST = [
$js_arr_items]" > ./src/hostlist.js
./release.sh

3
cert/.gitignore vendored Normal file
View File

@ -0,0 +1,3 @@
*
!.gitignore
!README.md

3
cert/README.md Normal file
View File

@ -0,0 +1,3 @@
该目录存放 HTTPS 证书，每个域名使用独立的目录。
证书通过 `server/gen-cert/run.sh` 生成。

BIN
docs/domain-model.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 359 KiB

BIN
docs/js-set-cookie.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 396 KiB

BIN
docs/login1.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 424 KiB

BIN
docs/login2.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 840 KiB

BIN
docs/sub-root-cookie.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 345 KiB

41
gen-cert/README.md Normal file
View File

@ -0,0 +1,41 @@
HTTPS 证书申请脚本
# 依赖
安装 acme.sh：
```bash
curl https://get.acme.sh | sh
```
# 生成
在当前目录下新建 `dnsconf` 文件，格式为：
```text
DOMAIN=example.com
DNS_ID=dns_xx
export xx_id=xxx
export xx_key=xxxxxx
```
第一个为域名，后面三个参考 https://github.com/Neilpang/acme.sh/wiki/dnsapi
例如 CloudFlare 的 DNS 服务：
```text
DOMAIN=etherdream.com
DNS_ID=dns_cf
export CF_Key="123456789012345678901234567890"
export CF_Email="user@gmail.com"
```
> API Keys 可在 https://dash.cloudflare.com/ 查看。
执行 `./gen.sh` 开始申请，证书文件保存到 `server/cert/域名` 目录下。
重启服务生效：
```bash
./server/run.sh reload
```

31
gen-cert/gen.sh Normal file
View File

@ -0,0 +1,31 @@
ACME=~/.acme.sh/acme.sh
source ./dnsconf
$ACME \
--issue \
--dns $DNS_ID \
-d *.$DOMAIN
$ACME \
--issue \
--dns $DNS_ID \
-d *.$DOMAIN \
--keylength ec-256
$ACME \
--install-cert -d *.$DOMAIN \
--key-file ../../server/cert/$DOMAIN/rsa.key \
--fullchain-file ../../server/cert/$DOMAIN/rsa.cer
$ACME \
--install-cert -d *.$DOMAIN --ecc \
--key-file ../../server/cert/$DOMAIN/ecc.key \
--fullchain-file ../../server/cert/$DOMAIN/ecc.cer
echo "
ssl_certificate cert/$DOMAIN/rsa.cer;
ssl_certificate_key cert/$DOMAIN/rsa.key;
ssl_certificate cert/$DOMAIN/ecc.cer;
ssl_certificate_key cert/$DOMAIN/ecc.key;
" > ../../server/cert/$DOMAIN/ngx.conf

58
nginx.conf Normal file
View File

@ -0,0 +1,58 @@
http {
server {
# server_name *.etherdream.com;
# listen 8443 ssl http2;
# include cert/etherdream.com/ngx.conf;
listen 8080;
include api.conf;
}
resolver 114.114.114.114 ipv6=off;
resolver_timeout 10s;
keepalive_timeout 60;
keepalive_requests 2048;
server_tokens off;
underscores_in_headers on;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-CHACHA20-POLY1305:ECDHE+AES128:RSA+AES128:ECDHE+AES256:RSA+AES256:ECDHE+3DES:RSA+3DES;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 5m;
ssl_prefer_server_ciphers on;
limit_req_log_level warn;
limit_req_zone $binary_remote_addr zone=reqip:16m rate=100r/s;
limit_req zone=reqip burst=200 nodelay;
# separated by tab (\t)
log_format log_proxy escape=none
'$time_iso8601 $_ver $remote_addr $upstream_cache_status $request_time '
'$request_length $bytes_sent '
'$request_method $_url $status $upstream_http_access_control_allow_origin '
'$http_user_agent'
;
access_log logs/proxy.log log_proxy buffer=64k flush=1s;
proxy_cache_path cache
levels=1:2
keys_zone=my_cache:8m
max_size=10g
inactive=6h
use_temp_path=off
;
proxy_http_version 1.1;
proxy_ssl_server_name on;
proxy_buffer_size 16k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_send_timeout 10s;
map $http_origin $_origin_allowed {
include allowed-sites.txt;
}
}
events {
worker_connections 4096;
}

1
server/nginx/.gitignore → nginx/.gitignore vendored
View File

@ -1,2 +1,3 @@
*
!logs
!.gitignore

0
server/cert/.gitignore → nginx/logs/.gitignore vendored
View File

14
run.sh Executable file
View File

@ -0,0 +1,14 @@
NGX_BIN=/home/jsproxy/openresty/nginx/sbin/nginx
# local test
if [ ! -f NGX_BIN ]; then
NGX_BIN=/usr/local/openresty/nginx/sbin/nginx
fi
CUR_DIR=$(cd `dirname $0` && pwd)
if [ $1 ]; then
PARAM="-s $1"
fi
$NGX_BIN -c $CUR_DIR/nginx.conf -p $CUR_DIR/nginx $PARAM

6
server/include/cert.conf
View File

@ -1,6 +0,0 @@
ssl_certificate cert/jsproxy.tk.fullchain.rsa.cer;
ssl_certificate_key cert/jsproxy.tk.rsa.key;
ssl_certificate cert/jsproxy.tk.fullchain.ecc.cer;
ssl_certificate_key cert/jsproxy.tk.ecc.key;

2
server/include/host-root.conf
View File

@ -1,2 +0,0 @@
server_name jsproxy.tk;
listen 443 ssl;

2
server/include/host-wild.conf
View File

@ -1,2 +0,0 @@
server_name *.jsproxy.tk;
listen 443 ssl;

131
server/include/rhost-vhost.map
View File

@ -1,131 +0,0 @@
default $_rhost_enc_ext.ext.jsproxy.tk;
google.com gg.jsproxy.tk;
www.google.com www.gg.jsproxy.tk;
*.google.com $_rhost_slice_1.gg.jsproxy.tk;
google.cn gc.jsproxy.tk;
www.google.cn www.gc.jsproxy.tk;
*.google.cn $_rhost_slice_1.gc.jsproxy.tk;
google.com.hk gk.jsproxy.tk;
www.google.com.hk www.gk.jsproxy.tk;
*.google.com.hk $_rhost_slice_2.gk.jsproxy.tk;
googleusercontent.com gu.jsproxy.tk;
www.googleusercontent.com www.gu.jsproxy.tk;
*.googleusercontent.com $_rhost_slice_1.gu.jsproxy.tk;
googlesource.com gs.jsproxy.tk;
www.googlesource.com www.gs.jsproxy.tk;
*.googlesource.com $_rhost_slice_1.gs.jsproxy.tk;
wikipedia.org wk.jsproxy.tk;
www.wikipedia.org www.wk.jsproxy.tk;
*.wikipedia.org $_rhost_slice_1.wk.jsproxy.tk;
m.wikipedia.org m.wk.jsproxy.tk;
www.m.wikipedia.org www.m.wk.jsproxy.tk;
*.m.wikipedia.org $_rhost_slice_2.m.wk.jsproxy.tk;
stackoverflow.com so.jsproxy.tk;
www.stackoverflow.com www.so.jsproxy.tk;
*.stackoverflow.com $_rhost_slice_1.so.jsproxy.tk;
stackexchange.com se.jsproxy.tk;
www.stackexchange.com www.se.jsproxy.tk;
*.stackexchange.com $_rhost_slice_1.se.jsproxy.tk;
serverfault.com sf.jsproxy.tk;
www.serverfault.com www.sf.jsproxy.tk;
*.serverfault.com $_rhost_slice_1.sf.jsproxy.tk;
superuser.com su.jsproxy.tk;
www.superuser.com www.su.jsproxy.tk;
*.superuser.com $_rhost_slice_1.su.jsproxy.tk;
askubuntu.com au.jsproxy.tk;
www.askubuntu.com www.au.jsproxy.tk;
*.askubuntu.com $_rhost_slice_1.au.jsproxy.tk;
github.com gh.jsproxy.tk;
www.github.com www.gh.jsproxy.tk;
*.github.com $_rhost_slice_1.gh.jsproxy.tk;
quora.com qr.jsproxy.tk;
www.quora.com www.qr.jsproxy.tk;
*.quora.com $_rhost_slice_1.qr.jsproxy.tk;
unix.com ux.jsproxy.tk;
www.unix.com www.ux.jsproxy.tk;
*.unix.com $_rhost_slice_1.ux.jsproxy.tk;
mozilla.org mz.jsproxy.tk;
www.mozilla.org www.mz.jsproxy.tk;
*.mozilla.org $_rhost_slice_1.mz.jsproxy.tk;
w3schools.com w3.jsproxy.tk;
www.w3schools.com www.w3.jsproxy.tk;
*.w3schools.com $_rhost_slice_1.w3.jsproxy.tk;
chromium.org cr.jsproxy.tk;
www.chromium.org www.cr.jsproxy.tk;
*.chromium.org $_rhost_slice_1.cr.jsproxy.tk;
myspace.com my.jsproxy.tk;
www.myspace.com www.my.jsproxy.tk;
*.myspace.com $_rhost_slice_1.my.jsproxy.tk;
facebook.com fb.jsproxy.tk;
www.facebook.com www.fb.jsproxy.tk;
*.facebook.com $_rhost_slice_1.fb.jsproxy.tk;
youtube.com yt.jsproxy.tk;
www.youtube.com www.yt.jsproxy.tk;
*.youtube.com $_rhost_slice_1.yt.jsproxy.tk;
twitter.com tw.jsproxy.tk;
www.twitter.com www.tw.jsproxy.tk;
*.twitter.com $_rhost_slice_1.tw.jsproxy.tk;
flickr.com fl.jsproxy.tk;
www.flickr.com www.fl.jsproxy.tk;
*.flickr.com $_rhost_slice_1.fl.jsproxy.tk;
reddit.com rd.jsproxy.tk;
www.reddit.com www.rd.jsproxy.tk;
*.reddit.com $_rhost_slice_1.rd.jsproxy.tk;
blogger.com bg.jsproxy.tk;
www.blogger.com www.bg.jsproxy.tk;
*.blogger.com $_rhost_slice_1.bg.jsproxy.tk;
wordpress.com wp.jsproxy.tk;
www.wordpress.com www.wp.jsproxy.tk;
*.wordpress.com $_rhost_slice_1.wp.jsproxy.tk;
medium.com md.jsproxy.tk;
www.medium.com www.md.jsproxy.tk;
*.medium.com $_rhost_slice_1.md.jsproxy.tk;
hackernoon.com hn.jsproxy.tk;
www.hackernoon.com www.hn.jsproxy.tk;
*.hackernoon.com $_rhost_slice_1.hn.jsproxy.tk;
yahoo.com yh.jsproxy.tk;
www.yahoo.com www.yh.jsproxy.tk;
*.yahoo.com $_rhost_slice_1.yh.jsproxy.tk;
bbc.com bc.jsproxy.tk;
www.bbc.com www.bc.jsproxy.tk;
*.bbc.com $_rhost_slice_1.bc.jsproxy.tk;
twitch.tv th.jsproxy.tk;
www.twitch.tv www.th.jsproxy.tk;
*.twitch.tv $_rhost_slice_1.th.jsproxy.tk;
steamcommunity.com sc.jsproxy.tk;
www.steamcommunity.com www.sc.jsproxy.tk;
*.steamcommunity.com $_rhost_slice_1.sc.jsproxy.tk;

131
server/include/vhost-rhost.map
View File

@ -1,131 +0,0 @@
*.ext.jsproxy.tk $_vhost_dec_ext;
gg.jsproxy.tk google.com;
www.gg.jsproxy.tk www.google.com;
*.gg.jsproxy.tk $_vhost_slice_0.google.com;
gc.jsproxy.tk google.cn;
www.gc.jsproxy.tk www.google.cn;
*.gc.jsproxy.tk $_vhost_slice_0.google.cn;
gk.jsproxy.tk google.com.hk;
www.gk.jsproxy.tk www.google.com.hk;
*.gk.jsproxy.tk $_vhost_slice_0.google.com.hk;
gu.jsproxy.tk googleusercontent.com;
www.gu.jsproxy.tk www.googleusercontent.com;
*.gu.jsproxy.tk $_vhost_slice_0.googleusercontent.com;
gs.jsproxy.tk googlesource.com;
www.gs.jsproxy.tk www.googlesource.com;
*.gs.jsproxy.tk $_vhost_slice_0.googlesource.com;
wk.jsproxy.tk wikipedia.org;
www.wk.jsproxy.tk www.wikipedia.org;
*.wk.jsproxy.tk $_vhost_slice_0.wikipedia.org;
m.wk.jsproxy.tk m.wikipedia.org;
www.m.wk.jsproxy.tk www.m.wikipedia.org;
*.m.wk.jsproxy.tk $_vhost_slice_1.m.wikipedia.org;
so.jsproxy.tk stackoverflow.com;
www.so.jsproxy.tk www.stackoverflow.com;
*.so.jsproxy.tk $_vhost_slice_0.stackoverflow.com;
se.jsproxy.tk stackexchange.com;
www.se.jsproxy.tk www.stackexchange.com;
*.se.jsproxy.tk $_vhost_slice_0.stackexchange.com;
sf.jsproxy.tk serverfault.com;
www.sf.jsproxy.tk www.serverfault.com;
*.sf.jsproxy.tk $_vhost_slice_0.serverfault.com;
su.jsproxy.tk superuser.com;
www.su.jsproxy.tk www.superuser.com;
*.su.jsproxy.tk $_vhost_slice_0.superuser.com;
au.jsproxy.tk askubuntu.com;
www.au.jsproxy.tk www.askubuntu.com;
*.au.jsproxy.tk $_vhost_slice_0.askubuntu.com;
gh.jsproxy.tk github.com;
www.gh.jsproxy.tk www.github.com;
*.gh.jsproxy.tk $_vhost_slice_0.github.com;
qr.jsproxy.tk quora.com;
www.qr.jsproxy.tk www.quora.com;
*.qr.jsproxy.tk $_vhost_slice_0.quora.com;
ux.jsproxy.tk unix.com;
www.ux.jsproxy.tk www.unix.com;
*.ux.jsproxy.tk $_vhost_slice_0.unix.com;
mz.jsproxy.tk mozilla.org;
www.mz.jsproxy.tk www.mozilla.org;
*.mz.jsproxy.tk $_vhost_slice_0.mozilla.org;
w3.jsproxy.tk w3schools.com;
www.w3.jsproxy.tk www.w3schools.com;
*.w3.jsproxy.tk $_vhost_slice_0.w3schools.com;
cr.jsproxy.tk chromium.org;
www.cr.jsproxy.tk www.chromium.org;
*.cr.jsproxy.tk $_vhost_slice_0.chromium.org;
my.jsproxy.tk myspace.com;
www.my.jsproxy.tk www.myspace.com;
*.my.jsproxy.tk $_vhost_slice_0.myspace.com;
fb.jsproxy.tk facebook.com;
www.fb.jsproxy.tk www.facebook.com;
*.fb.jsproxy.tk $_vhost_slice_0.facebook.com;
yt.jsproxy.tk youtube.com;
www.yt.jsproxy.tk www.youtube.com;
*.yt.jsproxy.tk $_vhost_slice_0.youtube.com;
tw.jsproxy.tk twitter.com;
www.tw.jsproxy.tk www.twitter.com;
*.tw.jsproxy.tk $_vhost_slice_0.twitter.com;
fl.jsproxy.tk flickr.com;
www.fl.jsproxy.tk www.flickr.com;
*.fl.jsproxy.tk $_vhost_slice_0.flickr.com;
rd.jsproxy.tk reddit.com;
www.rd.jsproxy.tk www.reddit.com;
*.rd.jsproxy.tk $_vhost_slice_0.reddit.com;
bg.jsproxy.tk blogger.com;
www.bg.jsproxy.tk www.blogger.com;
*.bg.jsproxy.tk $_vhost_slice_0.blogger.com;
wp.jsproxy.tk wordpress.com;
www.wp.jsproxy.tk www.wordpress.com;
*.wp.jsproxy.tk $_vhost_slice_0.wordpress.com;
md.jsproxy.tk medium.com;
www.md.jsproxy.tk www.medium.com;
*.md.jsproxy.tk $_vhost_slice_0.medium.com;
hn.jsproxy.tk hackernoon.com;
www.hn.jsproxy.tk www.hackernoon.com;
*.hn.jsproxy.tk $_vhost_slice_0.hackernoon.com;
yh.jsproxy.tk yahoo.com;
www.yh.jsproxy.tk www.yahoo.com;
*.yh.jsproxy.tk $_vhost_slice_0.yahoo.com;
bc.jsproxy.tk bbc.com;
www.bc.jsproxy.tk www.bbc.com;
*.bc.jsproxy.tk $_vhost_slice_0.bbc.com;
th.jsproxy.tk twitch.tv;
www.th.jsproxy.tk www.twitch.tv;
*.th.jsproxy.tk $_vhost_slice_0.twitch.tv;
sc.jsproxy.tk steamcommunity.com;
www.sc.jsproxy.tk www.steamcommunity.com;
*.sc.jsproxy.tk $_vhost_slice_0.steamcommunity.com;

2
server/include/x-js.conf
View File

@ -1,2 +0,0 @@
return 200 'importScripts("//jsproxy.tk/x.js")';

95
server/mime.types
View File

@ -1,95 +0,0 @@
types {
text/html html htm shtml;
text/css css;
text/xml xml;
image/gif gif;
image/jpeg jpeg jpg;
application/javascript js;
application/atom+xml atom;
application/rss+xml rss;
text/mathml mml;
text/plain txt;
text/vnd.sun.j2me.app-descriptor jad;
text/vnd.wap.wml wml;
text/x-component htc;
image/png png;
image/svg+xml svg svgz;
image/tiff tif tiff;
image/vnd.wap.wbmp wbmp;
image/webp webp;
image/x-icon ico;
image/x-jng jng;
image/x-ms-bmp bmp;
application/font-woff woff;
application/java-archive jar war ear;
application/json json;
application/mac-binhex40 hqx;
application/msword doc;
application/pdf pdf;
application/postscript ps eps ai;
application/rtf rtf;
application/vnd.apple.mpegurl m3u8;
application/vnd.google-earth.kml+xml kml;
application/vnd.google-earth.kmz kmz;
application/vnd.ms-excel xls;
application/vnd.ms-fontobject eot;
application/vnd.ms-powerpoint ppt;
application/vnd.oasis.opendocument.graphics odg;
application/vnd.oasis.opendocument.presentation odp;
application/vnd.oasis.opendocument.spreadsheet ods;
application/vnd.oasis.opendocument.text odt;
application/vnd.openxmlformats-officedocument.presentationml.presentation
pptx;
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
xlsx;
application/vnd.openxmlformats-officedocument.wordprocessingml.document
docx;
application/vnd.wap.wmlc wmlc;
application/x-7z-compressed 7z;
application/x-cocoa cco;
application/x-java-archive-diff jardiff;
application/x-java-jnlp-file jnlp;
application/x-makeself run;
application/x-perl pl pm;
application/x-pilot prc pdb;
application/x-rar-compressed rar;
application/x-redhat-package-manager rpm;
application/x-sea sea;
application/x-shockwave-flash swf;
application/x-stuffit sit;
application/x-tcl tcl tk;
application/x-x509-ca-cert der pem crt;
application/x-xpinstall xpi;
application/xhtml+xml xhtml;
application/xspf+xml xspf;
application/zip zip;
application/octet-stream bin exe dll;
application/octet-stream deb;
application/octet-stream dmg;
application/octet-stream iso img;
application/octet-stream msi msp msm;
audio/midi mid midi kar;
audio/mpeg mp3;
audio/ogg ogg;
audio/x-m4a m4a;
audio/x-realaudio ra;
video/3gpp 3gpp 3gp;
video/mp2t ts;
video/mp4 mp4;
video/mpeg mpeg mpg;
video/quicktime mov;
video/webm webm;
video/x-flv flv;
video/x-m4v m4v;
video/x-mng mng;
video/x-ms-asf asx asf;
video/x-ms-wmv wmv;
video/x-msvideo avi;
}

246
server/nginx.conf
View File

@ -1,246 +0,0 @@
user root;
worker_processes 1;
#error_log logs/error.log;
#error_log logs/error.log notice;
error_log logs/warn.log warn;
pid logs/nginx.pid;
events {
worker_connections 1024;
}
http {
# DNS 服务器地址
resolver 1.1.1.1 ipv6=off;
include mime.types;
default_type text/html;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 60;
keepalive_requests 1024;
server_tokens off;
more_clear_headers Server;
# 限流配置
limit_req_log_level warn;
limit_req_zone $binary_remote_addr zone=reqip:16m rate=100r/s;
limit_req zone=reqip burst=200 nodelay;
# 代理日志（分隔符 \t）
log_format log_proxy
'$time_iso8601 $remote_addr $request_time '
'$request_length $bytes_sent '
'$request_method $_proto $proxy_host $request_uri $status '
'$http_user_agent'
;
# 普通日志
log_format log_access
'$time_iso8601 $remote_addr $request_time '
'$request_method $uri $http_host $status '
'$http_user_agent'
;
access_log logs/access.log log_access buffer=64k flush=1s;
# 缓冲区配置
#（设置过低某些网站无法访问）
proxy_buffer_size 16k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_send_timeout 10s;
# 代理缓存配置
proxy_cache_path cache
levels=1:2
keys_zone=my_cache:8m
max_size=10g
inactive=6h
use_temp_path=off;
# SSL 双证书
include include/cert.conf;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-CHACHA20-POLY1305:ECDHE+AES128:RSA+AES128:ECDHE+AES256:RSA+AES256:ECDHE+3DES:RSA+3DES;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 5m;
proxy_cache my_cache;
proxy_http_version 1.1;
proxy_ssl_server_name on;
underscores_in_headers on;
merge_slashes off;
# 非内置域名的编码（将 `.` 替换成 `-dot-`）
# TODO: 由于 nginx map 不支持字符串替换，暂时先这么实现。。。
map $_rhost $_rhost_enc_ext {
volatile;
'~^([\w-]+)\.(\w+)$' '$1-dot-$2';
'~^([\w-]+)\.([\w-]+)\.(\w+)$' '$1-dot-$2-dot-$3';
'~^([\w-]+)\.([\w-]+)\.([\w-]+)\.(\w+)$' '$1-dot-$2-dot-$3-dot-$4';
}
# 非内置域名的解码（将 `-dot-` 替换成 `.`）
# TODO: 效率低而且级数有限，下次改成 lua 实现
map $_ext_src $_ext_dst {
volatile;
'~^([\w-]+?)-dot-([\w-]+?)-dot-([\w-]+?)-dot-([\w-]+?)-dot-([\w-]+?)-dot-(\w+)$'
'$1.$2.$3.$4.$5.$6';
'~^([\w-]+?)-dot-([\w-]+?)-dot-([\w-]+?)-dot-([\w-]+?)-dot-(\w+)$'
'$1.$2.$3.$4.$5';
'~^([\w-]+?)-dot-([\w-]+?)-dot-([\w-]+?)-dot-(\w+)$'
'$1.$2.$3.$4';
'~^([\w-]+?)-dot-([\w-]+?)-dot-(\w+)$'
'$1.$2.$3';
'~^([\w-]+?)-dot-(\w+)$'
'$1.$2';
}
map $_vhost $_vhost_dec_ext {
volatile; #.example.com
'~^(?<_ext_src>[^.]+)\.ext\.[\w-]+\.\w+$' $_ext_dst;
}
# 虚拟域名 -> 真实域名
map $_vhost $_vhost_to_rhost {
volatile;
hostnames;
include include/vhost-rhost.map;
}
# 真实域名 -> 虚拟域名
map $_rhost $_rhost_to_vhost {
volatile;
hostnames;
include include/rhost-vhost.map;
}
# TODO: 由于 hostnames 无法获取 * 部分，暂时先这样获取子域
# *.com -> *
map $_rhost $_rhost_slice_0 {
volatile;
'~^(.+?)\.\w+$' $1;
}
# *.google.com -> *
map $_rhost $_rhost_slice_1 {
volatile;
'~^(.+?)\.(?:[\w-]+\.){1}\w+$' $1;
}
# *.google.com.hk -> *
map $_rhost $_rhost_slice_2 {
volatile;
'~^(.+?)\.(?:[\w-]+\.){2}\w+$' $1;
}
# *.wk -> *
map $_vhost $_vhost_slice_0 {
volatile; #.example.com
'~^(.+?)\.\w+\.[\w-]+\.\w+$' $1;
}
# *.m.wk -> *
map $_vhost $_vhost_slice_1 {
volatile; #.example.com
'~^(.+?)\.(?:[\w-]+\.){1}\w+\.[\w-]+\.\w+$' $1;
}
##########
# 静态资源站点
server {
include include/host-root.conf;
root ../www;
charset utf-8;
add_header cache-control max-age=300;
add_header strict-transport-security 'max-age=99999999; includeSubDomains; preload';
# 需要编译 ngx_brotli 模块（参考 setup.sh）
brotli_static on;
}
# 内置站点代理
# 格式为 https://[sub.]alias.example.com/path/to/?query
server {
include include/host-wild.conf;
location / {
set $_vhost $host;
set $_site $_vhost_to_rhost;
if ($_site = '') {
return 404 "unknown site";
}
include proc-hdr.conf;
# 非 JS 发送的请求，返回安装 ServiceWorker 的页面
# 该请求为首次访问时发起，后续请求会被 SW 拦截并带上 JS 标记
if ($_hasSw = '0') {
rewrite ^ /__setup.html;
}
set $_proto 'https';
if ($_isHttp = '1') {
set $_proto 'http';
}
if ($_port) {
set $_port ':$_port';
}
# CORS preflight
if ($request_method = 'OPTIONS') {
more_set_headers
'access-control-allow-origin: $_acao'
'access-control-allow-Methods: GET, POST, PUT, DELETE, HEAD, OPTIONS'
'access-control-allow-Headers: $http_Access_Control_Request_Headers'
'access-control-max-Age: 1728000'
;
return 204;
}
# return 200 "[debug]
# request_uri: $request_uri
# host: $host
# isHttp: $_isHttp
# uri: $uri
# args: $args
# is_args: $is_args
# _ref: $_ref
# _ori: $_ori
# _hasSw: $_hasSw
# proxy: $_proto://$_site$_port;
# ";
access_log logs/proxy.log log_proxy buffer=64k flush=1s;
proxy_pass $_proto://$_site$_port;
# 将返回头 set-cookie 中的 domain 部分转换成我们的虚拟域名
proxy_cookie_domain ~^\.?(?<_rhost>.*)$ $_rhost_to_vhost;
}
location = /__setup.html {
internal;
brotli_static on;
charset utf-8;
root ../www;
etag off;
more_clear_headers Accept-Ranges Last-Modified;
}
# 由于 ServiceWorker 脚本必须位于同源站点，
# 因此为了减少重复加载，此处只返回实际脚本的引用。
location = /__sw.js {
add_header cache-control max-age=9999999;
include include/x-js.conf;
}
}
# 测试案例（暂未完成）
#include test.conf;
}

77
server/proc-hdr.conf
View File

@ -1,77 +0,0 @@
# 标记客户端是否已安装 Service Worker
# 0：请求任何路径，都返回 SW 安装页面（www/__setup.html）
# 1：正常反向代理
set $_hasSw '0';
# 标记资源的协议
# 0：HTTP
# 1：HTTPS
set $_isHttp '0';
# 标记是否为 CORS 请求
# 0：不转发 Origin 头
# 1：调整并转发 Origin 头
set $_hasCors '0';
# 记录资源的端口号
set $_port '';
set $_ref '';
set $_ori $http_origin;
set $_acao '';
# 获取并删除 flag 参数
# 参数格式: isHttp .. port
if ($args ~
(?<_pre>.*?)&flag__=(?<_hasSw>.)(?<_isHttp>.)(?<_hasCors>.)(?<_port>\d*)(?<_post>.*)
) {
set $args $_pre$_post;
set $_js 1;
}
# 调整 Referer 头
# TODO：未考虑协议和端口，下面的 cors 也有这问题
if ($http_referer ~ ^https://(?<_vhost>[^/]+)(?<_path>.*)) {
set $_ref https://$_vhost_to_rhost$_path;
set $_acao https://$_vhost;
}
if ($_ori) {
set $_acao $_ori;
}
if ($_acao = '') {
set $_acao '*';
}
# ServiceWorker 的 fetch 强制 cors 模式，
# 所以需要该标记，标识原始请求是否为 cors
if ($_hasCors = '0') {
set $_ori '';
}
if ($_ori ~ ^https://(?<_vhost>.*)) {
set $_ori https://$_vhost_to_rhost;
}
proxy_set_header Origin $_ori;
proxy_set_header Referer $_ref;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $http_connection;
# CSP 可能导致注入页面资源无法加载
more_clear_headers
content-security-policy
content-security-policy-report-only
expect-ct
x-frame-options
;
more_set_headers
'access-control-allow-credentials: true'
'access-control-allow-origin: $_acao'
'strict-transport-security: max-age=99999999; includeSubDomains; preload'
;
# 重定向调整
# 直接用 return 指令返回状态码，会丢失其他的头，比如 set-cookie
header_filter_by_lua_file ../proc-redir.lua;

49
server/proc-redir.lua
View File

@ -1,49 +0,0 @@
-- https://fetch.spec.whatwg.org/#statuses
local s = ngx.status
if not (s == 301 or s == 302 or s == 307 or s == 308) then
return
end
-- 忽略 WebSocket
if ngx.header['upgrade'] then
return
end
--[=[
如果直接返回 30X 状态，fetch API 会继续请求新的 URL，
不符合 req.redirect 为 manual 的情况。
例如请求 google.com 会重定向到 www.google.com，
如果最终获得的内容是后者，但地址栏显示的是前者，路径上就会出现问题。
如果在 SW 里设置 req.redirect = manual，重定向后拿不到 location。
所以这里对状态码 + 10 进行转义，SW 收到后再 -10。
]=]
ngx.status = s + 10
ngx.header['access-control-expose-headers'] = 'location'
-- local url = ngx.header['location']
-- if not url then
-- return
-- end
-- -- m = [, rhost, path]
-- local r = [[^https?://([^/]+)(.*)]]
-- local m = ngx.re.match(url, r, 'jo')
-- if not m then
-- return
-- end
-- -- rhost to vhost
-- ngx.var._rhost = m[1]
-- local vhost = ngx.var._rhost_to_vhost
-- url = 'https://' .. vhost .. m[2]
-- -- add flag
-- local sign = url:find('?', 1, true) and '&' or '?'
-- url = url .. sign .. 'flag__=' .. ngx.var._flag
-- -- update redir url
-- ngx.header['location'] = url

8
server/run.sh
View File

@ -1,8 +0,0 @@
NGX_BIN=/usr/local/openresty/nginx/sbin/nginx
CUR_DIR=$(cd `dirname $0` && pwd)
if [ $1 ]; then
PARAM="-s $1"
fi
$NGX_BIN -c $CUR_DIR/nginx.conf -p $CUR_DIR/nginx $PARAM

62
server/setup.sh
View File

@ -1,62 +0,0 @@
# nodejs
curl -sL https://rpm.nodesource.com/setup_10.x | sudo bash -
yum install -y \
gcc gcc-c++ clang \
zlib zlib-devel unzip \
git bc sed tree \
make autoconf automake libtool \
nodejs
npm i -g webpack webpack-cli
npm i -g html-minifier
# install openresty
mkdir -p install
cd install
curl -O https://ftp.pcre.org/pub/pcre/pcre-8.42.zip
unzip pcre-*
curl -O https://www.openssl.org/source/openssl-1.0.2p.tar.gz
tar zxvf openssl-*
git clone --recurse-submodules --depth 1 https://github.com/google/ngx_brotli.git
curl -O https://openresty.org/download/openresty-1.13.6.2.tar.gz
tar zxvf openresty-*
cd openresty-*
export NGX_BROTLI_STATIC_MODULE_ONLY=1
./configure \
--add-module=../ngx_brotli \
--with-http_ssl_module \
--with-openssl=../openssl-1.0.2p \
--with-pcre=../pcre-8.42 \
--with-pcre-jit
make
make install
# install brotli
# https://www.howtoforge.com/how-to-compile-brotli-from-source-on-centos-7/
git clone --depth 1 https://github.com/google/brotli.git
cd ./brotli
cp docs/brotli.1 /usr/share/man/man1 && gzip /usr/share/man/man1/brotli.1
./bootstrap
./configure --prefix=/usr \
--bindir=/usr/bin \
--sbindir=/usr/sbin \
--libexecdir=/usr/lib64/brotli \
--libdir=/usr/lib64/brotli \
--datarootdir=/usr/share \
--mandir=/usr/share/man/man1 \
--docdir=/usr/share/doc
make
make install
# install acme.sh
curl https://get.acme.sh | sh

1
server/www/__setup.html
View File

@ -1 +0,0 @@
<p id=t></p><script>function n(){var e=Date.now();try{if(e-(+sessionStorage._ts||0)<100)return setTimeout(n,2e3);sessionStorage._ts=e}catch(t){}location.reload()}function r(e){t.innerHTML=e.message}top===self&&(t.innerHTML="loading...");var o=navigator.serviceWorker;o&&self.ReadableStream?o.getRegistration().then(function(e){e?n():o.register("/__sw.js").then(n)["catch"](r)}):t.innerHTML="请使用最新版 Chrome 浏览器访问"</script>

BIN
server/www/__setup.html.br
View File

Binary file not shown.

26
server/www/index.html
View File

@ -1,26 +0,0 @@
<!doctype html>
<html>
<head>
<title>JS Proxy</title>
<meta charset="utf-8">
<script src="x.js"></script>
<style>
#txtURL {
width: 300px;
}
</style>
</head>
<body>
<h1>网页沙盒</h1>
<div>
URL:
<input id="txtURL" value="https://www.google.com.hk">
<button id="btnGo">Go</button>
</div>
<script>
btnGo.onclick = function() {
open(txtURL.value)
}
</script>
</body>
</html>

231
server/www/x.js
View File

File diff suppressed because one or more lines are too long

BIN
server/www/x.js.br
View File

Binary file not shown.

33
setup-ipset.sh Executable file
View File

@ -0,0 +1,33 @@
# run as root
ipset create ngx-ban-dstip hash:net
iptables \
-A OUTPUT \
-p tcp --syn \
-m owner --uid-owner jsproxy \
-m set --match-set ngx-ban-dstip dst \
-j REJECT
# https://en.wk.jsproxy.tk/wiki/Reserved_IP_addresses
REV_NET=(
0.0.0.0/8
10.0.0.0/8
100.64.0.0/10
127.0.0.0/8
169.254.0.0/16
172.16.0.0/12
192.0.0.0/24
192.0.2.0/24
192.88.99.0/24
192.168.0.0/16
198.18.0.0/15
198.51.100.0/24
203.0.113.0/24
224.0.0.0/4
240.0.0.0/4
255.255.255.255/32
)
for v in ${REV_NET[@]}; do
ipset add ngx-ban-dstip $v
done

16
setup-nginx.sh Executable file
View File

@ -0,0 +1,16 @@
#
curl -O https://openresty.org/download/openresty-1.15.8.1rc1.tar.gz
tar zxvf openresty-*
cd openresty-*
./configure \
--with-http_v2_module \
--with-http_ssl_module \
--with-pcre-jit \
--prefix=/home/jsproxy/openresty
make
make install
cd ..
rm -rf openresty-*

32
sitelist.txt
View File

@ -1,32 +0,0 @@
gg google.com
gc google.cn
gk google.com.hk
gu googleusercontent.com
gs googlesource.com
wk wikipedia.org
m.wk m.wikipedia.org
so stackoverflow.com
se stackexchange.com
sf serverfault.com
su superuser.com
au askubuntu.com
gh github.com
qr quora.com
ux unix.com
mz mozilla.org
w3 w3schools.com
cr chromium.org
my myspace.com
fb facebook.com
yt youtube.com
tw twitter.com
fl flickr.com
rd reddit.com
bg blogger.com
wp wordpress.com
md medium.com
hn hackernoon.com
yh yahoo.com
bc bbc.com
th twitch.tv
sc steamcommunity.com

19
upload.sh Executable file
View File

@ -0,0 +1,19 @@
HOST=etherdream.com
NODE=(
node-aliyun-hk
node-aliyun-sg
)
for v in ${NODE[@]}; do
echo "$v upload ..."
rsync . jsproxy@$v.$HOST:server \
--delete -r \
--exclude='nginx/cache/*' \
--exclude='nginx/logs/*'
echo "$v restart ..."
ssh jsproxy@$v.$HOST "./server/run.sh reload"
done
echo "done"