Uploading huge IPA fails

[eRadical]

Hi,

We're trying to upload a 1G ipa file (little over actually).

We have nginx in front of the docker containers to account for HTTPS.
Nginx limits are set to huge numbers but the upload does not go through... the same happens if I try to address the MobDistTool (on another port).

Is there any max file size baked into MobDistTool?

[rgroult]

Hi, do you have some backtraces on error logs on that ?

[eRadical]

Hi @rgroult ,

Unfortunately we're using the docker image.

Is there any way of enabling debug in that?

[rgroult]

Hi,
Use docker log to retrieve output or get the log file (see your conf about the log directory) inside your docker image.
There is no way to start in debug mode yet.

[eRadical]

After some debugging we ended up to:

2017-07-19 14:04:27.588066 0:00:00.015572 GET [200] /api/in/v1/artifacts/9e4b4a93-220d-4224-a00a-812241ca226a/deploy
tcmalloc: large alloc 1073750016 bytes == (nil) @ 0xb0b64a 0x987ed8 0x8e11d5 0x8e0b50 0x8e1166 0x8e2bd2 0x8e2d50 0xad1d09 0x7fd58aa30579 0x7fd58910e547 0x7fd58910e467 0x7fd58911cc45 0x7fd584d7062a 0x7fd584d6f353 0x7fd585863a33 0x7fd58587f110 0x7fd58587e96f 0x7fd58587b851 0x7fd589123fda 0x7fd589123d7d 0x7fd58587e3da 0x7fd58911617e 0x7fd589123589 0x7fd589123455 0x7fd58587e3da 0x7fd5840bcd6e 0x7fd5840bcbc0 0x7fd58585f66e 0x7fd5840bc38f 0x7fd5840c2455 0x7fd584317660
[2017-07-19 14:04:52.220599] INFO RPC: RPC Error with status: 400 and message: Unable to add artifact: Unable to analyse IPA
2017-07-19 13:56:15.903621 0:08:36.321892 POST [400] /api/in/v1/artifacts/9e4b4a93-220d-4224-a00a-812241ca226a/master/v2.4.2/RELEASE

Together w/ a colleague we've looked into artifact_analyzer.dart and there are a lot things that may end with "Unable to analyse IPA".

I'll try to add some swap to the machine to see if it helps!
I'll keep you posted.

[eRadical]

After adding 4G of swap... out of which he used little over 2G it cored dumped:

/dart_runtime/dart_run.sh: line 23: 5 Aborted (core dumped) /usr/bin/dart ${DBG_OPTION} --enable-vm-service:8181/0.0.0.0 ${DART_VM_OPTIONS} bin/server.dart

Is there anyway (in dart) to change lines 27 to 31 (from artifact_analyzer.dart) to use a call to a system provided unzip package?

Example:
unzip -p TheGame.ipa Payload/*.app/Info.plist > Local-Info.plist

Then you can read the Local-Info.plist file.

[rgroult]

hummm.. yep it not really optimized (not at all :S) for very large file. Usually we use smaller IPAs, I didn't test with this kind of file. I will search another way..

[eRadical]

Same here... from 11 games (in the platform) only 1 has this size & this issue.

Thanx for the awesome work in MobDistTool.

[rgroult]

Hi,
can you try the docker latest image with new env key MDT_IPA_EXTRACT_USING_UNZIP="true" in environnement ? It allows using unzip system call instead of "dart unzipping". My current network connection is too small to test docker image :S
Revert on last stable image if any problem.

[eRadical]

Unfortunaelly..

tcmalloc: large alloc 2147491840 bytes == 0x7a2a000 @ 0xb0b64a 0x987ed8 ........
[2017-07-21 08:59:11.625383] INFO RPC: RPC Error with status: 400 and message: Unable to add artifact: writeFrom failed

Image details
$docker images | awk '/mobdisttool/ && /latest/'
rgroult2/mobdisttool latest ceb24fbc69f8 13 hours ago 446MB

And it's still using over 2G of swap space.

[rgroult]

arff.. the problem is in the RPC framework, It store received file temporaly and then send it to MDT.
I need to check inside RPC framework.