Skip to content
This repository was archived by the owner on Aug 30, 2025. It is now read-only.
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 47 additions & 1 deletion src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@ extern crate serde_json;

use chrono::{DateTime, Utc};
use reqwest::header::{qitem, Accept, Authorization, Bearer, Link, RelationType, UserAgent};
use reqwest::StatusCode;
use serde_derive::Deserialize;
use std::time::{UNIX_EPOCH, SystemTime, Duration};
use std::{error, fmt, mem};

#[derive(Debug)]
Expand Down Expand Up @@ -99,22 +101,57 @@ impl fmt::Display for Repository {

pub fn collect_stars(config: Config) -> Result<(), Box<dyn error::Error>> {
let mut builder = ClientBuilder::new();
let mut print_auth_warning = false;

if let Some(ref token) = config.token {
builder.set_authorization_token(token.to_owned());
}
else {
print_auth_warning = true;
}

let client = builder.build()?;

let mut stars: Vec<Star> = Vec::new();

let mut next_link = config.url();

let mut remaining: i32 = 0;
let mut total: i32 = 0;
let mut mins = 0;

while next_link.is_some() {
if let Some(link) = next_link {
let mut res = client.get(&link).send()?;
next_link = extract_link_next(res.headers());

for header in res.headers().iter() {
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you're iterating the map of Header values here. Is this because reqwest doesn't implement headers for the X-RateLimit- headers GitHub is returning to us?

If you're up for it/interested, we could implement the Header trait for these X-RateLimit- headers so we can get() the individual Headers based on their type. If not I totally understand, it's something we can turn into an issue for later improvement.

Copy link
Copy Markdown
Author

@biskit1 biskit1 Nov 29, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's right. I've opened up issue #36 to implement the Header trait, and will change this code around once that is completed. Didn't realize I can do that, so thanks for the suggestion and link.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't necessarily have to be part of this pull request, the current approach works just fine.

I'd actually encourage that we leave #36 as a later enhancement and finish getting this pull request merged in, since implementing custom Headers isn't critical to getting these warnings implemented.

match header.name() {
"X-RateLimit-Limit" => {
total = header.value_string().parse::<i32>()?;
},
"X-RateLimit-Remaining" => {
remaining = header.value_string().parse::<i32>()?;
},
"X-RateLimit-Reset" => {
let seconds = header.value_string().parse::<u64>()?;

// Creates a new SystemTime from the specified number of whole seconds
let reset_time = UNIX_EPOCH + Duration::from_secs(seconds);
mins = reset_time.duration_since(SystemTime::now())?.as_secs()/60;
},
_ =>(),
}
}

match res.status() {
StatusCode::Forbidden => {
return Err(format!("Uh-oh! You have {} out of {} requests remaining. Your request limit will reset in {} minutes.", remaining, total, mins).into());
},
_ => (),
}

next_link = extract_link_next(res.headers());

let mut s: Vec<Star> = res.json()?;
stars.append(&mut s);
}
Expand All @@ -125,6 +162,15 @@ pub fn collect_stars(config: Config) -> Result<(), Box<dyn error::Error>> {
}
println!("Collected {} stars", stars.len());

if print_auth_warning {
eprintln!("\nRequest completed without authentication, re-run and provide token using `--token <auth-token>` to increase your requests from 60 to 5000 requests per hour.");
}

match remaining {
10 | 0...5 => eprintln!("Warning: You have {} out of {} requests remaining. Your request limit will reset in {} minutes.", remaining, total, mins),
_ => (),
}

Ok(())
}

Expand Down