Sunshine
59a8be493d
add support for working with local assets
5 years ago
Sunshine
d5ee8ae6ab
account for legacy BODY background="" attribute
5 years ago
Sunshine
7654eec7e2
treat frames the same way as iframes
5 years ago
Sunshine
0d1e21e9ad
add black box tests
5 years ago
Sunshine
3d2d40e7cd
add support for data URL targets
5 years ago
Sunshine
b8b6d8cff6
fix "succeeding" to "passing" in tests
5 years ago
Sunshine
928664dc88
correct is_valid_url to is_http_url
5 years ago
Sunshine
5c8d75539b
rename dataurl to data_url
5 years ago
Sunshine
d9602e25eb
update help dialog and README.md
5 years ago
Sunshine
f720fe0176
Merge pull request #114 from snshn/custom-network-timeout-option
...
Add option for custom network request timeout
5 years ago
Sunshine
727a5a410c
add option for custom network request timeout
5 years ago
Sunshine
5ef2b7c9dc
nullify JS within As' href attributes when needed
5 years ago
rhysd
0e12cecd85
Revert "Merge pull request #106 from rhysd/proxy-support"
...
This reverts commit d8def879b2
, reversing
changes made to a9d114d04d
.
5 years ago
rhysd
d47482fcd9
fix crash at setting empty values to HTTP proxies
...
with this patch `https_proxy=` and `http_proxy=` will work well.
5 years ago
rhysd
b68624f2f3
support HTTP and HTTPS proxies ( fix #103 )
5 years ago
Sunshine
a9d114d04d
Merge pull request #105 from rhysd/refactor-main
...
Refactoring for main.rs to address several issues
5 years ago
rhysd
4e4ebe9c98
refactor main to address several issues
...
Addressed issues:
- when specified URL is invalid, it exited successfully with doing
nothing. There was no way why it does not work for users
- it exited successfully even if invalid User-Agent value is specified
- it created file twice on `--output` option specified. It may cause an
issue when some file watcher (e.g. FsEvents on macOS) is watching
Improvements:
- handle errors with `Result::expect` consistently it correctly exits
with non-zero status on error
- define `Output` enum for handling both stdout and file outputs
5 years ago
rhysd
1779f4a374
better comments for JS_DOM_EVENT_ATTRS constant
5 years ago
rhysd
26e89ae6d3
use complete list of DOM event handlers
5 years ago
rhysd
69d99b69e8
remove . in line comment
5 years ago
Emi Simpson
05985583f0
Switch timestamps from rfc822 local time to iso8601 UTC
5 years ago
Emi Simpson
651fa716b4
Clean user, pass, and fragment from URL before writing
5 years ago
rhysd
67b79e92f9
simplify &x.into_iter() to x.iter()
5 years ago
rhysd
b51f41fe34
trim attribute values
5 years ago
rhysd
6f158dc6db
compare value of 'rel' properties in case-insensitive
5 years ago
rhysd
8d7052b39c
ignore preload and prefetch sources
...
since all resources are embedded as data URL.
5 years ago
rhysd
660511b8a0
define link type of <link> element as enum and prefer match statement
...
since match statement checks exhaustiveness
5 years ago
Emi Simpson
9be3982dc6
Added --no-context flag to disable adding context comment
5 years ago
Emi Simpson
27c9fb4cd3
Added comment indicating the context under which the page was downloaded
5 years ago
rhysd
6e99ad13e7
upgrade reqwest to v0.10.0
...
This will improve build time and binary size as follows:
* Before
- **Compile targets**: 220
- **Build time**: `cargo build --release 1264.95s user 39.72s system 335% cpu 6:29.14 total`
- **Binary size**: 6578568 bytes
* After
- **Compile targets**: 170
- **Build time**: `cargo build --release 1130.64s user 32.15s system 359% cpu 5:23.69 total`
- **Binary size**: 6107088 bytes
* Differences
- **Compile targets**: 1.29x smaller
- **Build time**: 1.23x faster
- **Binary size**: 1.07x smaller
5 years ago
Sunshine
413dd66886
Merge pull request #96 from rhysd/refactorings
...
Refactorings
5 years ago
rhysd
dc7ec6e7a8
remove more redundant type annotations
5 years ago
rhysd
ed879231af
fix test code was broken by refactoring
5 years ago
rhysd
ddf4b8ac13
prefer &str to String for reducing allocations
5 years ago
rhysd
84c13f0605
prefer unwrap_or_default to unwrap_or
5 years ago
rhysd
ce03e0e487
reduce allocation on checking DOM attributes and do not hard-code number of elements of array constant
...
`to_lower` allocates new string but the allocation is not necessary
here.
5 years ago
rhysd
63e19998d0
reduce clones and fix some code styles and redundant code
5 years ago
rhysd
75fb6961ed
migrate to Rust 2018
5 years ago
Sunshine
5ba8931502
Merge pull request #92 from snshn/output-file-option
...
Add option for saving output to file
5 years ago
Sunshine
88ffde0c3b
wipe integrity attributes
5 years ago
Sunshine
bfb97bd062
add option for saving output to file
5 years ago
Sunshine
295931041c
Merge pull request #80 from Alch-Emi/lazyload
...
Add support for lazy loaded images
5 years ago
Emi Simpson
dab4ae6965
Merged Y2Z/master with Alch-Emi/lazyload
5 years ago
Sunshine
c7fc121c7c
use clean URLs as hashmap keys
5 years ago
Sunshine
9ff9dd0928
Merge pull request #82 from snshn/str
...
implement str!() macro
5 years ago
Emi Simpson
3d4a932ac1
Merge Y2Z/master, fix conflicts between shared-client & resolve-css
5 years ago
Sunshine
9fe913d853
implement str!() macro
5 years ago
Sunshine
862489e41b
Get rid of brackets around URLs
5 years ago
Emi Simpson
322ab41b8c
Updated tests to reflect API changes
5 years ago
Emi Simpson
65d0eab793
Use a shared client initialized in main.rs
5 years ago
Emi Simpson
292221ea28
Lazyloaded images are now loaded at compilation, with placeholders omitted
5 years ago
Emi Simpson
614af44c92
Gramatical and stylistic fixes
5 years ago
Emi Simpson
feb37f5812
Added support for lazy loaded images
...
Note: The way this patch works is by resolving any data-src tags on images in
the same way as normal source tags are resolved. It is assumed that most
lazy-load libraries will use this tag, and that if this tag is set, then it is a
URL that is in use.
5 years ago
Emi Simpson
028beb821c
Rustfmt update for nightly formatter
5 years ago
Emi Simpson
76ccff80f9
Fixed failure of regex to match @imports
5 years ago
Emi Simpson
45335d7507
Support links in style= attributes
5 years ago
Emi Simpson
a4743ca92f
Respect the --no-images flag while parsing CSS
5 years ago
Emi Simpson
b96a777e8a
Merge commit '4decea7' into load-css-imports
5 years ago
Emi Simpson
4decea716c
Fixed css replacement with more than one linked asset
5 years ago
Emi Simpson
695a787206
Moved regex compilation to lazy_static
5 years ago
Emi Simpson
90e6cb1c45
Prevent crash on URLs delimited by single quotes
5 years ago
Emi Simpson
7412d663e0
Use a slightly more efficient .replace_range() instead of cloning the string twice
5 years ago
Emi Simpson
8646af6e9f
removed debug code (woops sorry)
5 years ago
Emi Simpson
de383c94b1
Applied rustfmt
5 years ago
Emi Simpson
ab65b44f0d
Cleaned up some overcomplicated code
5 years ago
Emi Simpson
13bacb4320
EMPTY_STRING no longer used
5 years ago
Emi Simpson
d574e9a5da
Added support for <style> tags
5 years ago
Emi Simpson
1de0fc0961
Add warning and fallback when parsing a rel=stylesheet link
5 years ago
Emi Simpson
ebbf755e09
Fixed misleading variable name
5 years ago
Emi Simpson
d3956a7905
Made merge compatible with Y2Z/master
5 years ago
Emi Simpson
ef7ddcd434
Added fallback to absolute URL on failure to resolve CSS stylesheet @imports
5 years ago
Emi Simpson
11bbfc0851
Added support for recursively nested css @imports
5 years ago
Emi Simpson
a2bf7e3345
Fixed some errors detecting, parsing, and transforming urls in `resolve_css_imports`
5 years ago
voila
1ff5e91087
Use HashMap as cache to minimize the number of HTTP requests ( #75 )
...
Use HashMap as cache to minimize the number of HTTP requests
5 years ago
knidarkness
550e4cc83f
Fixed formatting
5 years ago
knidarkness
5443c0cc3f
Added loading of the links given as url(...) in css files
5 years ago
robatipoor
55fe523a1c
refactor utils functions
5 years ago
robatipoor
2e48ea90e1
move argument parser section to args mod
5 years ago
Sunshine
0896f2e214
Properly handle 30x redirects
5 years ago
Vincent Flyson
3948ea3aa0
Improve code structure
5 years ago
Vincent Flyson
eec05767cf
Add support for poster attribute
5 years ago
Vincent Flyson
88a230872c
Add CSP isolation, no CSS, and no iframe options
5 years ago
Vincent Flyson
04cbbefafa
Ignore empty src in images, accept fluid icons
5 years ago
Vincent Flyson
824b418f80
Add more tests
5 years ago
Vincent Flyson
2251c086c2
Merge branch 'master' into ignore-ca
5 years ago
Vincent Flyson
02b717ae54
Add flag to ignore errors related to TLS certificates
5 years ago
Vincent Flyson
1329dbe6f8
Ignore iframes with empty src
5 years ago
Vincent Flyson
2b96f9a32a
Add flag for silent mode
5 years ago
Vincent Flyson
50f1ba1ce8
Treat network errors as empty results
5 years ago
Vincent Flyson
0b2e1d0746
Remove compiler warning
5 years ago
Vincent Flyson
2c0037fd51
Merge pull request #37 from Y2Z/ignore-other-protocols
...
Avoid modifying non-HTTP anchor hrefs
5 years ago
Vincent Flyson
3f79068d7d
Avoid modifying non-HTTP anchor hrefs
5 years ago
Vincent Flyson
080ed50264
Merge pull request #36 from Y2Z/no-images-no-icons
...
Ignore icons if told to avoid images
5 years ago
Vincent Flyson
b596b5bb7d
Ignore icons if told to avoid images
5 years ago
Vincent Flyson
c480fcb70f
Add basic verbose output
5 years ago
Vincent Flyson
e2ab05a323
Fix name of a test function
5 years ago
Vincent Flyson
d49e825777
Add support for picture elements
5 years ago
Vincent Flyson
d8d6437a15
Merge branch 'master' into author-robatipoor
5 years ago
Vincent Flyson
c34d77d5d8
Revamp resolve_url() and improve code format
5 years ago
Vincent Flyson
2be5b1c235
Add support for iframes
5 years ago