Skip to content

Conversation

@wangzhao0217
Copy link
Contributor

  • Update batch.R
  • Update batch.R
  • Update batch.R
  • update

@wangzhao0217
Copy link
Contributor Author

@Robinlovelace we could keep this changes locally for further test

@Robinlovelace
Copy link
Collaborator

@wangzhao0217 the output of batch() is now noisy:

> library(sf)
Linking to GEOS 3.12.1, GDAL 3.8.4, PROJ 9.3.1; sf_use_s2() is TRUE
> desire_lines = od::od_to_sf(od::od_data_df, od::od_data_zones)[4:5, 1:3]
0 origins with no match in zone ids
0 destinations with no match in zone ids
 points not in od data removed.
> u1 = "https://github.com/cyclestreets/cyclestreets-r/"
> u2 = "releases/download/v0.5.3/od-longford-10-test.Rds"
> u = paste0(u1, u2)
> desire_lines = readRDS(url(u))
> routes_id = batch(desire_lines, username = "robinlovelace", wait = FALSE)
Error in batch(desire_lines, username = "robinlovelace", wait = FALSE) : 
  could not find function "batch"
> devtools::load_all()
ℹ Loading cyclestreets
> routes_id = batch(desire_lines, username = "robinlovelace", wait = FALSE)
POSTing the request to create and start the job
-> Host: api.cyclestreets.net
-> User-Agent: libcurl/7.81.0 r-curl/5.2.3 httr/1.4.7
-> Accept-Encoding: deflate, gzip, br, zstd
-> Accept: application/json, text/xml, application/xml, */*
-> Content-Length: 2641
-> Content-Type: multipart/form-data; boundary=------------------------6eea3cfb113898a3
-> 

>> Content-Disposition: form-data; name="name"
>> 
>> Batch job

>> Content-Disposition: form-data; name="serverId"
>> 
>> 21

>> Content-Disposition: form-data; name="geometry"
>> 
>> {"type":"FeatureCollection","features":[{"type":"Feature","properties":{"id":1},"geometry":{"type":"LineString","coordinates":[[-7.79885,53.72181],[-7.80357,53.72463]]}},{"type":"Feature","properties":{"id":2},"geometry":{"type":"LineString","coordinates":[[-7.79866,53.72565],[-7.79833,53.72215]]}},{"type":"Feature","properties":{"id":3},"geometry":{"type":"LineString","coordinates":[[-7.80779,53.7258],[-7.78967,53.72503]]}},{"type":"Feature","properties":{"id":4},"geometry":{"type":"LineString","coordinates":[[-7.7897,53.72285],[-7.80064,53.72518]]}},{"type":"Feature","properties":{"id":5},"geometry":{"type":"LineString","coordinates":[[-7.8051,53.72636],[-7.79534,53.72656]]}},{"type":"Feature","properties":{"id":6},"geometry":{"type":"LineString","coordinates":[[-7.79914,53.73046],[-7.80609,53.72401]]}},{"type":"Feature","properties":{"id":7},"geometry":{"type":"LineString","coordinates":[[-7.8024,53.7203],[-7.79575,53.73139]]}},{"type":"Feature","properties":{"id":8},"geometry":{"type":"LineString","coordinates":[[-7.78989,53.72459],[-7.80175,53.72742]]}},{"type":"Feature","properties":{"id":9},"geometry":{"type":"LineString","coordinates":[[-7.80759,53.72897],[-7.79016,53.72604]]}},{"type":"Feature","properties":{"id":10},"geometry":{"type":"LineString","coordinates":[[-7.79698,53.7243],[-7.7953,53.72223]]}}]}
>> Content-Disposition: form-data; name="bothDirections"
>> 
>> 0
...
>> Content-Disposition: form-data; name="maxDistance"
>> 
>> 5000
...
>> Content-Disposition: form-data; name="includeJsonOutput"
>> 
>> 1
...
>> Content-Disposition: form-data; name="username"
>> 
>> robinlovelace
...
>> Content-Disposition: form-data; name="password"
...

<- HTTP/1.1 200 OK
<- Date: Wed, 04 Dec 2024 10:30:12 GMT
<- Server: Apache
<- Upgrade: h2
<- Connection: Upgrade
...
<- Expires: Mon, 26 Jul 1997 05:00:00 GMT
<- Cache-Control: no-store, no-cache, must-revalidate
<- Pragma: no-cache
<- Access-Control-Allow-Origin: *
<- Last-Modified: Wed, 04 Dec 2024 10:30:12 GMT
<- Cache-Control: post-check=0, pre-check=0
<- Content-Length: 134
<- Content-Type: application/json; charset=UTF-8
<- 
Job id: 11948
...
-> Accept: application/json, text/xml, application/xml, */*
-> Content-Length: 358
...
-> 
...
>> Content-Disposition: form-data; name="username"
>> 
>> robinlovelace
...
>> Content-Disposition: form-data; name="password"
...

<- HTTP/1.1 200 OK
<- Date: Wed, 04 Dec 2024 10:30:12 GMT
<- Server: Apache
<- Expires: Mon, 26 Jul 1997 05:00:00 GMT
<- Cache-Control: no-store, no-cache, must-revalidate
<- Pragma: no-cache
<- Access-Control-Allow-Origin: *
<- Last-Modified: Wed, 04 Dec 2024 10:30:12 GMT
<- Cache-Control: post-check=0, pre-check=0
<- Content-Length: 125
<- Content-Type: application/json; charset=UTF-8
<- 
Routing job sent, check back in around 0 minutes if you've just sent this
Check at www.cyclestreets.net/journey/batch/ for route id: 11948
> routes_wait = batch(id = routes_id, username = "robinlovelace", wait = TRUE, delete_job = FALSE)
-> POST /v2/batchroutes.jobdata?key=... HTTP/1.1
-> Host: api.cyclestreets.net
-> User-Agent: libcurl/7.81.0 r-curl/5.2.3 httr/1.4.7
-> Accept-Encoding: deflate, gzip, br, zstd
...-> Accept: application/json, text/xml, application/xml, */*
-> Content-Length: 358
...
-> 
...
>> Content-Disposition: form-data; name="id"
>> 
>> 11948
...
>> Content-Disposition: form-data; name="username"
>> 
...
>> 
...
>> --------------------------066b258e15bd2552--

<- HTTP/1.1 200 OK
<- Date: Wed, 04 Dec 2024 10:30:59 GMT
<- Server: Apache
<- Upgrade: h2
<- Connection: Upgrade
<- Expires: Mon, 26 Jul 1997 05:00:00 GMT
<- Cache-Control: no-store, no-cache, must-revalidate
<- Pragma: no-cache
<- Access-Control-Allow-Origin: *
<- Last-Modified: Wed, 04 Dec 2024 10:30:59 GMT
<- Cache-Control: post-check=0, pre-check=0
<- Content-Length: 125
<- Content-Type: application/json; charset=UTF-8
<- 
No data returned yet. Trying again id 11948 every 10 seconds

Copy link
Collaborator

@Robinlovelace Robinlovelace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make the output less noisy?

@Robinlovelace
Copy link
Collaborator

There are 3 more verbose() calls that should be removed or commented out.

That' cool

Co-authored-by: Robin Lovelace <Robinlovelace@users.noreply.github.com>
R/batch.R Outdated
pause_min = 30, # Min delay between retries
terminate_on = NULL, # Don't terminate on specific status codes
retry_on = NULL, # Retry on all errors (since connection refused isn't an HTTP status code)
httr::verbose() # Optional: outputs detailed request info
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remove verbose here and in the other places?

@Robinlovelace Robinlovelace merged commit d6a6111 into cyclestreets:master Dec 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants