Tom Parker-Shemilt
|
3b592dae23
|
Simplified handles with tokio Semaphore
|
2020-06-03 18:05:12 +01:00 |
|
Tom Parker-Shemilt
|
4b7e6d1488
|
Get handle once per recursive sequence of get_url calls
|
2020-06-03 17:57:07 +01:00 |
|
Tom Parker-Shemilt
|
5e9357ed6f
|
Swap back to older reqwest API
|
2020-06-02 23:02:18 +01:00 |
|
Tom Parker-Shemilt
|
87ad03cb15
|
Remove handle debug because it overloads travis
|
2020-06-02 22:40:28 +01:00 |
|
Tom Parker-Shemilt
|
a1abde30b9
|
Upgrade packages and change over deprecated Reqwest function
|
2020-06-02 22:32:16 +01:00 |
|
Tom Parker-Shemilt
|
c680c63f8a
|
Disable connection pooling, as it seems to break stuff
|
2020-06-02 22:25:01 +01:00 |
|
Tom Parker-Shemilt
|
25084b928a
|
Various debug for checks
|
2020-06-02 22:24:49 +01:00 |
|
Tom Parker-Shemilt
|
1f6cb3749b
|
Panic earlier on 404'ed links
|
2020-05-01 00:22:48 +01:00 |
|
Tom Parker-Shemilt
|
6532f7681f
|
Only do Github API rewrite if have environment variables
|
2020-04-30 22:04:01 +01:00 |
|
Tom Parker-Shemilt
|
2dfb07854f
|
Merge pull request #867 from palfrey/add-github-secrets
Support optional use of Github secrets to workaround rate limits
|
2020-04-25 18:26:03 +01:00 |
|
Tom Parker-Shemilt
|
1efa0de042
|
Missed two other cases of needing to print url with link
|
2020-04-21 11:18:53 +01:00 |
|
Tom Parker-Shemilt
|
e953dbb269
|
Print url before Link
|
2020-04-21 11:04:59 +01:00 |
|
Tom Parker-Shemilt
|
0e3bada631
|
Support optional use of Github secrets to workaround rate limits (fixes #859)
|
2020-04-03 22:10:09 +01:00 |
|
Tom Parker-Shemilt
|
15dbb3602b
|
For most errors, intermittent failures are ok, not so much for redirects
|
2020-04-03 21:12:54 +01:00 |
|
Tom Parker-Shemilt
|
60e614f51e
|
Nicer "failure only occured X ago" message
|
2020-03-26 23:29:34 +00:00 |
|
Tom Parker-Shemilt
|
d4b44fd6d5
|
Don't remove last_working if a link fails
|
2020-03-26 23:22:40 +00:00 |
|
Tom Parker-Shemilt
|
12464c2844
|
Redo (not-)working with a enum, not a pair of variables
|
2020-03-10 21:17:59 +00:00 |
|
Tom Parker-Shemilt
|
592a4bed92
|
Output all the info on link failure
|
2020-03-08 16:49:14 +00:00 |
|
Tom Parker-Shemilt
|
edc17508e1
|
Only write every so often
|
2020-03-08 16:49:04 +00:00 |
|
Tom Parker-Shemilt
|
2901a666cb
|
Merge branch 'master' into cache-results
|
2020-03-03 23:10:02 +00:00 |
|
Tom Parker-Shemilt
|
f3ac7ba5d3
|
Merge pull request #840 from palfrey/move-cleanup
Move cleanup into bin folder
|
2020-03-03 23:08:37 +00:00 |
|
Tom Parker-Shemilt
|
0d798c3896
|
Github workflows have started to do 406, unless we add an explicit image to the Accept header
|
2020-02-26 21:26:18 +00:00 |
|
Tom Parker-Shemilt
|
43eab5fa9a
|
Merge branch 'master' into better-unknown-travis-detection
|
2020-02-25 21:26:29 +00:00 |
|
Tom Parker-Shemilt
|
9d41a10148
|
Cache the results of getting links across Travis build and only re-check
every so often
|
2020-02-24 23:41:45 +00:00 |
|
Tom Parker-Shemilt
|
d649b575a7
|
Update cleanup instructions
|
2020-02-24 22:38:19 +00:00 |
|
Tom Parker-Shemilt
|
e6520cbac0
|
Move cleanup to bin folder and run it on the repo
|
2020-02-24 22:36:50 +00:00 |
|
Tom Parker-Shemilt
|
ecd6658ec6
|
Improve detection of "unknown" travis builds
|
2020-02-16 20:44:45 +00:00 |
|
Tom Parker-Shemilt
|
0a527ff6e3
|
Catch no-branch cases for Travis
|
2020-02-12 23:38:26 +00:00 |
|
Tom Parker-Shemilt
|
5bceef3656
|
Catch github actions badges with no branch
|
2020-02-12 23:32:02 +00:00 |
|
Tom Parker-Shemilt
|
89efda92e3
|
Azure devops does 302 twice, so need to cope with that
|
2020-02-10 21:02:28 +00:00 |
|
Tom Parker-Shemilt
|
4d048eba27
|
Add workaround for Youtube 302s (fixes #814)
|
2020-01-30 15:23:42 +00:00 |
|
Tom Parker-Shemilt
|
dec99f4416
|
Also look for travis.org as well as .com
|
2020-01-20 18:05:20 +00:00 |
|
Tom Parker-Shemilt
|
f0af98f3fc
|
Detect unknown status travis builds
|
2020-01-20 17:53:24 +00:00 |
|
Tom Parker-Shemilt
|
7ea1063c32
|
Update user-agent so Reddit stops complaining
|
2020-01-20 13:50:02 +00:00 |
|
Tom Parker-Shemilt
|
6cb9433de0
|
Look for links/images in HTML content in the Markdown
|
2020-01-18 14:01:05 +00:00 |
|
Tom Parker-Shemilt
|
a37b1cfee3
|
Redo error checking with failure and less verbose formatting
|
2020-01-17 23:24:03 +00:00 |
|
Tom Parker-Shemilt
|
5733288c9a
|
Record actual real errors
|
2020-01-13 22:21:03 +00:00 |
|
Tom Parker-Shemilt
|
f799381f75
|
Add 20 second timeout
|
2020-01-13 22:13:13 +00:00 |
|
Tom Parker-Shemilt
|
c25072ec06
|
Add workaround for github actions
|
2020-01-13 22:13:00 +00:00 |
|
Tom Parker-Shemilt
|
ff703bac03
|
Fix accept header so we also get github svgs
|
2020-01-13 21:53:25 +00:00 |
|
Tom Parker-Shemilt
|
a6b79d73ec
|
Also pick up image tags
|
2020-01-13 17:04:46 +00:00 |
|
Tom Parker-Shemilt
|
fd843a5dd8
|
Add extra debug on redirects
|
2020-01-13 16:17:24 +00:00 |
|
Tom Parker-Shemilt
|
39b2e4b3b6
|
Add text/hmtl header to fix crates.io
|
2020-01-13 16:07:48 +00:00 |
|
Tom Parker-Shemilt
|
70361602dd
|
Detect redirects
|
2020-01-13 08:39:38 +00:00 |
|
Tom Parker-Shemilt
|
67bfa13da5
|
Retry up to 5 times just in case on failure
|
2020-01-11 14:45:49 +00:00 |
|
Tom Parker-Shemilt
|
f6e1302d17
|
Limit running gets to 20 to stop issues with file handles
|
2020-01-11 12:06:40 +00:00 |
|
Tom Parker-Shemilt
|
57a7f96ba5
|
Start writing a rust-based url checker
|
2020-01-10 23:07:36 +00:00 |
|
kud1ing
|
0d11f0b0c5
|
remove
|
2017-05-03 18:46:46 +02:00 |
|
kud1ing
|
4b22d82e49
|
merge the deployment sections
|
2017-04-29 10:00:25 +02:00 |
|
kud1ing
|
2978c44acc
|
add section 'Text search'
|
2016-12-28 17:32:40 +01:00 |
|