Kevin Lynx
|
82f14c458c
|
optimize highlight for http
|
11 years ago |
|
Kevin Lynx
|
c0b383a7b7
|
add http search result highlight
|
11 years ago |
|
Kevin Lynx
|
2f8842a18d
|
fix stats 1970 bug
|
11 years ago |
|
Kevin Lynx
|
53e3036f4f
|
add sphinx config doc
|
11 years ago |
|
Kevin Lynx
|
fa0b80908a
|
update readme
|
11 years ago |
|
Kevin Lynx
|
4a9b85c973
|
fix sphinx builder query range bug; improve sphinx builder db query performance, to avoid `getmore' command and the 4M useless response
|
11 years ago |
|
Kevin Lynx
|
f887208cd5
|
log bug
|
11 years ago |
|
Kevin Lynx
|
ba64278d12
|
build date index at startup
|
11 years ago |
|
Kevin Lynx
|
648fe7f20d
|
daman! fix crawler initiali id generation bug (not set random seed)
|
11 years ago |
|
Kevin Lynx
|
377912982d
|
add some debug log to sphinx_builder, test sphinx_builder when there's no hashes there and got new hashes
|
11 years ago |
|
Kevin Lynx
|
7af23fcc49
|
add log level config for sphinx_builder
|
11 years ago |
|
Kevin Lynx
|
db83eecfd5
|
change sphinx_builder, query from mongodb by `skip' really don't work well, build a date index to query by date range
|
11 years ago |
|
Kevin Lynx
|
6f6aac3b35
|
adjust crawler log directory
|
11 years ago |
|
Kevin Lynx
|
f16d25dae7
|
http ui stuff
|
11 years ago |
|
Kevin Lynx
|
60bb12538e
|
http ui adjust
|
11 years ago |
|
Kevin Lynx
|
aed757f2a8
|
modify `giza' library so that i can get sphinx search stats, and because of this, i can add a more detailed page navigation
|
11 years ago |
|
Kevin Lynx
|
237d90f81a
|
add a new config `search_method', if set to `sphinx', hash reader will not create name_array, also add config for httpd, to config the search method
|
11 years ago |
|
Kevin Lynx
|
486c354ba0
|
change sphinx torrent loading using an existing cursor
|
11 years ago |
|
Kevin Lynx
|
92826bf848
|
turn off the damn debug log
|
11 years ago |
|
Kevin Lynx
|
79291ab4e9
|
fix sphinx xml utf8 related issure, filter these unicode control characters, only backup delta file if the operation failed
|
11 years ago |
|
Kevin Lynx
|
1d870e2e42
|
add sphinx search stats
|
11 years ago |
|
Kevin Lynx
|
1d27f2416b
|
add a simple page navigation for sphinx_search
|
11 years ago |
|
Kevin Lynx
|
e1c905b0a7
|
fix sphinx_build memory leak bug, caused by mongo_cursor
|
11 years ago |
|
Kevin Lynx
|
46c99cabd8
|
sphinx worker call infinity
|
11 years ago |
|
Kevin Lynx
|
7b1a435a43
|
fix some sphinx related bugs, now it can be used to build sphinx index, still in experiment stage, add `giza' library to query sphinx in http_fontend
|
11 years ago |
|
Kevin Lynx
|
7ab79b5d2e
|
variable name change
|
11 years ago |
|
Kevin Lynx
|
e5011ab75a
|
fix sphinx doc creation failed
|
11 years ago |
|
Kevin Lynx
|
f242d4e44f
|
add sphinx support, in expirment status right now
|
11 years ago |
|
Kevin Lynx
|
b961dc9c46
|
add LICENSE.txt, lincensed by MIT
|
11 years ago |
|
Kevin Lynx
|
ec456de63d
|
fix daterange issure which not only record today torrents, not it only show the today inserted torrents
|
11 years ago |
|
Kevin Lynx
|
4dc05bf2cc
|
adjust http stats display
|
11 years ago |
|
Kevin Lynx
|
cb914fe609
|
system stats adjust, add more stats to http front-end
|
11 years ago |
|
Kevin Lynx
|
2a9f99940a
|
add a new force to string log func, add log to httpd, it can log unicode characters to logfiles
|
11 years ago |
|
Kevin Lynx
|
3b0e5701c8
|
complete all http uri to json api
|
11 years ago |
|
Kevin Lynx
|
980c6cad57
|
add query stats for new hash_writer
|
11 years ago |
|
Kevin Lynx
|
e46c264056
|
add `size' function to hash_download_cache, to debug
|
11 years ago |
|
Kevin Lynx
|
108a1bfd1b
|
fix hash_download_cache startup bug
|
11 years ago |
|
Kevin Lynx
|
dcf0181839
|
NOTE: rewrite hash_reader, config changed, dht_hash database changed, require to remove existed dht_hash database
|
11 years ago |
|
Kevin Lynx
|
75b3d82f4c
|
change default config
|
11 years ago |
|
Kevin Lynx
|
060804ae31
|
fix cache_indexer bug
|
11 years ago |
|
Kevin Lynx
|
2658040f3a
|
add simple `get' json api, fix http search space decode
|
11 years ago |
|
Kevin Lynx
|
54a30122fa
|
fix hash_date
|
11 years ago |
|
Kevin Lynx
|
37ccb19575
|
change hash_date only record the new inserted torrents
|
11 years ago |
|
Kevin Lynx
|
76542be37a
|
config max download task per hash-reader
|
11 years ago |
|
Kevin Lynx
|
4882cbf692
|
fix hash-writer cache-writting issure
|
11 years ago |
|
Kevin Lynx
|
cd9ae2ec53
|
Merge branch 'src' of github.com:kevinlynx/dhtcrawler2 into src
|
11 years ago |
|
Kevin Lynx
|
5592b3989b
|
fix hash_reader stop working bug
|
11 years ago |
|
Kevin Lynx
|
01451534ee
|
change crawler to cache hashes and merge hashes before inserted into db
|
11 years ago |
|
Kevin Lynx
|
1dc4a2c588
|
fix cache_indexer download error
|
11 years ago |
|
Kevin Lynx
|
75841f8237
|
fix cache_indexer state not saved correctly
|
11 years ago |