iOS packet tunnel provider limit #4422
Comments
|
on the iOS(with 8 GB ram) and in the "streisand" app, in addition to Loyalsoldier's geo-ip and geo-site, I added another geo-site(iran.dat), but still no problem occurred. |
|
@patterniha The RAM here is irrelevant; this is specifically about the packet tunnel provider. Are you sure you’re using the latest files? As of now, the old v2fly files are still included by default in Streisand. Which categories are you using? For 100% of users in the control group, categories like geosite:category-ads-all, geosite:cn, and others caused the tunnel to crash. |
|
geoip:ir, geosite:category-ads, geosite:category-ir, ext:iran.dat:tld-ir Although I did this about a month ago and now I don't have access to an iPhone to test again. |
|
Do you mean that the problem is with the large size of the loyalsoldier and the previous default geo has no problem? |
|
Xray-core 做过优化只解析有需要的 categories，不过前提是先把整个 geo 文件加载到内存中，我看了下新的是 21.3MB 如果这部分要继续优化就只能逐块读取文件 |
|
@patterniha I have created two simple visualizers for new files: You can check the categories you specified. In this case, with all due respect, your comparison is not relevant. The The |
|
还有我觉得 iOS 这种内存受限的设备上倒不必跟着 Xray-core 换 geofiles，应该去用一些精简版的 geofiles |
我不懂这个 inspiration from neighboring teams 指的是什么，Xray 早在 v1.1.4 就有了优化机制，v2fly 是抄 Xray 的做法但只字未提 https://github.com/XTLS/Xray-core/releases/tag/v1.1.4 https://github.com/v2fly/v2ray-core/releases/tag/v4.39.0 不过我刚看了下 v2fly 这个是直接对着 file 读而不是对着内存中的 file 读，可以省点瞬时内存，但仍不敌 unmarshal 后的内存占用 要想 unmarshal 后占的内存少点，只能使用精简版的 geofiles 还是说你指的是 sing-box 的 rule-sets？那个我还没看 |
|
"utilizes multiple smaller files instead of one monolithic file" |
|
其实不切文件也能先用 1MB 的缓存遍历整个文件并建立 index 索引的，然后要哪部分就读哪部分，如果 Golang 方便这么做的话 |
|
但是这样也就顶多省个瞬时内存，比如减小 20MB 这样，对 unmarshal 后还是没有帮助，还是需要用精简版的 geofiles 如果是为了控制在 iOS 的 50MB 限制内，最终还是不能使用大而全的 geofiles，“进一步省瞬时内存”的必要性就存疑了 |
|
因为即使启动阶段省下了 20MB，后面代理跑起来也需要内存、缓存，给个 20MB 不过分吧 如果“进一步省瞬时内存”后，启动阶段都吃了顶格 50MB 了，后面还有什么内存跑代理 |
|
这么说来现在的一次性把 21.3MB 加载到内存中更像是一种“探测”，如果这都跑不起来，即使先放你通过，后面肯定动不动就 crash |
|
@Fangliding |
|
if the problem is related to premise load of geofile --> It should crash even for smaller categories like "ir". so the problem is related to size of big categories like "cn" or "ads-all". /// but 99% of the domains are less than 32 bytes length. so why does it occupy 21.3 MB? |
|
21.3MB 是现在 geoip.dat 14.7MB 加 geosite.dat 6.6MB，不过它俩本身就有“压缩”，unmarshal 后会膨胀一些，没测算过膨胀的比例 不过 Xray 不是整个 unmarshal，而是先把这 21.3MB 读进内存，找到目标 categories 再 unmarshal，然后释放这 21.3MB，再跑代理 继续优化就是不一次性把 21.3MB 读进内存，但如我所说这样没什么意义，即使能先在 iOS 上跑起来了，后面跑代理的内存也不富裕 |
|
这里有数据：#4348 (comment) 但我也说过：#4348 (comment) 我觉得倒是可以看看 unmarshal 后的结构有没有优化空间 |
why 27m and 36m? So even if we assume that each domain is 32 bytes after decompression(although 99% of them are shorter) --> |
看了下，果然发现了大问题： Xray-core/app/router/config.pb.go Lines 138 to 150 in be43f66 Xray-core/app/router/config.pb.go Lines 203 to 213 in be43f66
|
|
@Fangliding 你试试把前三个 member 给删了并编译出来看看能少吃多少内存，还有 Domain Attribute 是干嘛的，实际有用到吗 而且它们的数组还是 |
|
我试了下，直接把前三个 member 删了都能正常使用 geofiles |
|
attr是分组里的tag |
|
这个rule的大小是小问题吧 最后都要被编译成它AC自动机里的trie树的 |
|
在路由使用 "geosite:cn" 的情况下，我的配置优化前静置内存占 15.1MB，优化后静置内存占 14.7MB 但是我发现了一个更严重的问题，如果 "geosite:cn" 出现了两次，内存占用会增加约 8MB 这应该是因为 |
应该是因为关缓存只需关 geofiles 的缓存，而不应该关 unmarshal 后的内容的缓存，结果当初一起关了
不过按 @Fangliding 的说法，如果这里 rule 的大小无关，可能是 GC 不及时的问题？因为优化前后确实有区别 并且产生了新问题，如果像 还有我在想 unmarshal 时 IP 那个 []byte 应该是 clone 而不是取现有文件的切片吧。。。 |
|
|
|
在 libXray 的早期版本中，实现了一种优化方案：在 packet tunnel 启动前，对用到的 geo 文件进行裁剪（只保留用到的规则）。具体可参见 geo_cut.go 。该文件已在后续更新中被删除。 如果你们正在开发或改进你们的 iOS 客户端，此方案可以作为一种参考。 Domain Attribute 通常用于域名规则的特定地区分流，比如“geosite:apple@cn”，则只作用于 apple 相关的中国区域名。 In early versions of libXray, an optimization solution was implemented: before starting the packet tunnel, the geo files used were trimmed (only the rules used were retained). For details, please see geo_cut.go. This file has been removed in a subsequent update. If you are developing or improving your iOS client, this solution can be used as a reference. |
|
When I worked with loading large files (100+ GB) over the network in Java, I used file streaming to disk instead of loading into RAM. Then I processed those large compressed files without unpacking (the uncompressed ones were 600-1000 GB). Is it possible to use the streaming principle in Xray? To restructure the file so that it can be read not entirely into RAM, but in chunks consisting of byte arrays or whatever is inside the .dat file? Store meta information about the sections inside the file and access specific areas instead of reading everything and then searching for the required part. The CPU load will be higher, but why not? |
|
Do I understand correctly that if the rules specify domain:ru, there will be no pupa.ru, zalupa.ru in the RAM? |
|
geosite:zalupa Inside pupa.ru, ru How will the core behave? |
|
@RPRX Hello. My problem is not exactly on-topic, but it also concerns the 50MB memory limit on iOS. When running the xhttp configuration (xhttp-reality, xhttp-tls, xhttp-tls-cdn) with default “extra” settings on iOS, memory usage exceeds 50MB at its peak on any client. I observed on a Mac that the app can consume up to 96MB. And this happens even with routing completely disabled. In the case of xhttp-reality, using I check this by running the Speedtest app or opening the ipleak.net site. In these cases, memory usage grows significantly above the set limit. So, here’s my question. Could you please advise if there are any client configuration settings that could help maintain stable xhttp usage on iOS? |
even with simple "ws" transport and not using geofiles, many users report that the VPN is constantly disconnected and they have to manually turn it on again.(now I realize it was because of the RAM.) we are in 2025 and the age of artificial intelligence, but iOS has set a 50MB limit even on 8GB series!!! |
|
所以到底怎么办 在三个tg群发个投票? |
https://raw.githubusercontent.com/Loyalsoldier/geoip/release/geoip-only-cn-private.dat |
还是不行， 看多几个视频内存就超了。 我使用控制变量法分别测试了hev-sock5-tunnel和xray, 第一种 第二种, 但是只要tunnel和xray都在ios中运行的时候, 内存就很快会超出. 况且我都分别针对tunnel和xray做了内存限制, 但是并没什么效果. |
|
说实话就是 iOS 这个 50MB 的限制真的落后，但凡 100MB 都没这问题，什么年代了都， 具体到对内存占用的优化，大概等到我写的客户端上 iOS 时我才有动力，不然 core 优化、iOS 客户端卖钱，我闲的吗 所以现阶段要推动优化内存这件事更应当由各个 iOS 客户端来研究，欢迎 PR，或者你们找到哪里该优化，我直接写代码也很快 |
|
Xray 刚出前两年没什么 iOS 客户端，就是因为限制 15MB，后来改成了 50MB，iOS 客户端就雨后春笋一样出来了 2025 年了，是时候改成 100MB 了，我寻思全局就这一个 VPN 也不差这 50MB，或许你们给苹果发个邮件，提醒它更新下内存限制 |
|
|
|
以我开发时长两年半的经验来看，我觉得还是弃疗吧。对于 Xray-core，一般客户端的集成方案都是 tun2socks + Xray-core。这个方案里，tun2socks 通常要占用 10M。 还是等 ”尊贵的苹果大人“ 放开限制吧。 |
|
经过这几天的测试我发现xray在ios15和ios16上的内存是无法及时释放的，但是ios17以上就没这个问题。 难道是要放弃ios17以下了？ |
我也觉得苹果这个限制是够奇葩的了。但是有什么办法呢。 |
|
关于 不过我发现了一个优化点是 DomainMatcher 可以复用，即 "geosite:cn" 出现多次时不会额外增加内存占用 还有 RoutingRule 里的 |
GeoIPMatcher 也应当被复用，不过现在的问题是路由那里写的是数组，然后不同来源的规则被揉到了一起，想复用的话需要拆开放 |
|
以及 GeoIPMatcher 的结构感觉有点复杂了，这里应该有优化空间，至少能把内存占用优化掉 3/10 |
我这几天的测试好像都是用全局测试的， 下面是我测试用的配置。 按理跟统计相关东西应该不太会影响内存吧。 { }, |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
VPN连接成功的时候，内存占用在23M，这个时候没有访问任何网站的话，会一直稳定在这个， 打开第一个网站whoer.net. 内存占用就火速标升超过50M |
|
先叠个甲，我对xray的实现以及iOS开发都不太明了，但这个问题我确实遇到了，我自己用了一个解法，使得无需裁剪dat文件。 |
没错的，现在一般都是这么操作的， 把文件还有配置文件那些放到group目录中去。 |
|
libXray has added ThinGeoData function. Maybe it will help to reduce memory usage on start. |
|
根据我的测试，当域名规则总量超过6000，VPN 内存占用更容易超过限制。所以如果你们为客户提供默认规则配置，可以适当控制用到的规则分类。 According to my test, when the total number of domain rules exceeds 6000, the VPN memory usage is more likely to exceed the limit. So if you provide customers with default rule configurations, you can appropriately control the rule categories used. |
你不懂资本主义自由市场,苹果的系统 库克想怎么样就怎么样,欧盟政府没有权力管.EU极左死了,严重扰乱市场. |
|
|
ip这块绝大多数客户端都在用cn、ir、ru这几个有墙国家的tag 实际上这个issues只要让geoip把ir ru也像cn一样打包个单独的dat出来基本就够了 为此可以提交一个PR到geoip，只要改动几行就行非常简单 感觉没必要让每个xray实例都自己去割geodat，感觉代码会变得挺脏，应该fork geodat直接分割好 目前得显式指定切割好的文件来用 |
|
|
|
I tested the latest version of Xray ( happ , streinsand ) without any routing on iOS device (iPhone 16 Pro Max) After uploading 20–30 MB, the core crashes. The config uses xhttp tls xmux "maxConcurrency": "4-8" and WS over tls mux "concurrency": 8 |
Original discussion: #4398
Good afternoon, dear members of our community!
Specifically, we know for certain that in the current versions of iOS, we have the following memory limitations:
In our case, Xray is a network process limited to 50MB. It reads geo files and then operates with the read content.
Unfortunately, with the current implementation of geofiles in our core, where all categories are in one large file, we often encounter the inability to even simply start the tunnel on an Apple device.
In particular, with the new geolocation files that we received from our partners, the file sizes have become significantly larger, which means that we cannot load them into memory.
A simple example (Loyalsoldier geosite):
None of the first four categories, if set in the routing rules, will allow the tunnel to start.
This also applies to geo files with CIDR (Loyalsoldier geoip)
Simple example:
@RPRX , considering the memory constraints, perhaps we should explore refactoring the geofile handling code. We could draw inspiration from neighboring teams and implement a system that utilizes multiple smaller files instead of one monolithic file. This approach could effectively resolve the memory issues on Apple devices. What you think about this?
The text was updated successfully, but these errors were encountered: