[Bug]: lowcoder-node-service: PayloadTooLargeError: request entity too large

[dobromyslov]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Error in lowcoder-node-service logs:

POST /node-service/api/npm/registry/lowcoder-comps 500 4.918 ms - 38
GET /node-service/api/plugins 200 68.087 ms - 59233
PayloadTooLargeError: request entity too large
    at readStream (/lowcoder/node-service/app/node_modules/raw-body/index.js:163:17)
    at getRawBody (/lowcoder/node-service/app/node_modules/raw-body/index.js:116:12)
    at read (/lowcoder/node-service/app/node_modules/body-parser/lib/read.js:79:3)
    at jsonParser (/lowcoder/node-service/app/node_modules/body-parser/lib/types/json.js:138:5)
    at /lowcoder/node-service/app/build/server.js:42:64
    at newFn (/lowcoder/node-service/app/node_modules/express-async-errors/index.js:16:20)
    at Layer.handle [as handle_request] (/lowcoder/node-service/app/node_modules/express/lib/router/layer.js:95:5)
    at trim_prefix (/lowcoder/node-service/app/node_modules/express/lib/router/index.js:328:13)
    at /lowcoder/node-service/app/node_modules/express/lib/router/index.js:286:9
    at Function.process_params (/lowcoder/node-service/app/node_modules/express/lib/router/index.js:346:12)
    at next (/lowcoder/node-service/app/node_modules/express/lib/router/index.js:280:10)
    at urlencodedParser (/lowcoder/node-service/app/node_modules/body-parser/lib/types/urlencoded.js:103:7)
    at newFn (/lowcoder/node-service/app/node_modules/express-async-errors/index.js:16:20)
    at Layer.handle [as handle_request] (/lowcoder/node-service/app/node_modules/express/lib/router/layer.js:95:5)
    at trim_prefix (/lowcoder/node-service/app/node_modules/express/lib/router/index.js:328:13)
    at /lowcoder/node-service/app/node_modules/express/lib/router/index.js:286:9 {
  expected: 61,
  length: 61,
  limit: 50,
  type: 'entity.too.large'
}
POST /node-service/api/npm/registry/lowcoder-comps 500 0.791 ms - 38

lowcoder-node-service uses LOWCODER_MAX_REQUEST_SIZE to configure the upload limit at:

lowcoder/server/node-service/src/server.ts

Line 33 in ec6539d

const MAX_REQUEST_SIZE = process.env.LOWCODER_MAX_REQUEST_SIZE || "50mb";

The default vaule of LOWCODER_MAX_REQUEST_SIZE is 20m in the default environment config:

lowcoder/deploy/docker/default.env

Line 120 in ec6539d

LOWCODER_MAX_REQUEST_SIZE=20m

Expected Behavior

In my opinion, there should be different request size limits:

• One existing LOWCODER_MAX_REQUEST_SIZE for general use across the system (to limit end-users uploads)
• Additional LOWCODER_NODE_SERVICE_MAX_REQUEST_SIZE=75m one for lowcoder-node-service internal requirements to deal with huge NPM libraries.

The default value of MAX_REQUEST_SIZE should be increased at least to 75m at:

lowcoder/server/node-service/src/server.ts

Line 33 in ec6539d

const MAX_REQUEST_SIZE = process.env.LOWCODER_MAX_REQUEST_SIZE || "50mb";

I tried 50m - does not work.

Steps to reproduce

1. Run lowcoder v2.7.5 as docker-compose-multi
2. Try to create and publish some App
3. See the logs of the lowcoder-node-service docker container

Environment

v2.7.5 docker-compose-multi

Additional Information

It's been discussed on Discord at https://discord.com/channels/1096896040159957084/1424088366328057866/1426537985784021114, however I could not find any changes for this problem.

Workaround: set LOWCODER_MAX_REQUEST_SIZE=75m in the override.dev file and restart lowcoder-node-service

After applying the workaround:

POST /node-service/api/npm/registry/lowcoder-comps 200 176.025 ms - 279911
{"level":30,"time":1760894881725,"pid":506,"hostname":"491e45e90c52","msg":"Fetch registry info for path: /lowcoder-comps"}
{"level":30,"time":1760894881725,"pid":506,"hostname":"491e45e90c52","msg":"No registry entry found for package: lowcoder-comps"}
POST /node-service/api/npm/registry/lowcoder-comps 200 142.015 ms - 279911

[ScottKemp]

Falk mentioned in discord that "mb" is the best suffix to use, not "m"

[dobromyslov]

Actually, NGinx does not support "mb", it requires "m"

• https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size
• https://nginx.org/en/docs/syntax.html

Sizes and offsets

Sizes can be specified in bytes, kilobytes, or megabytes using the following suffixes:

• k and K for kilobytes
• m and M for megabytes

For example, “1024”, “8k”, “1m”.
Offsets can be also specified in gigabytes using the g or G suffixes.

So "mb" could be better to use only if we change the ".sh" scripts that create NGinx configs inside the docker containers.

[ScottKemp]

Uggg. Lets hope this gets fixed. We have been fighting this for months.