Miroslav Stampar
|
2a4a284a29
|
crawler fix (skip binary files)
|
2011-06-20 22:41:38 +00:00 |
|
Miroslav Stampar
|
20bb1a685b
|
really minor update
|
2011-06-20 21:57:53 +00:00 |
|
Miroslav Stampar
|
812cd2f19b
|
minor update
|
2011-06-20 21:47:03 +00:00 |
|
Miroslav Stampar
|
e8ac7414f2
|
bug fix
|
2011-06-20 21:36:15 +00:00 |
|
Miroslav Stampar
|
d6062e8fc9
|
minor fix for crawler and far less message overlaps in future
|
2011-06-20 21:18:12 +00:00 |
|
Miroslav Stampar
|
8968c708a0
|
minor update
|
2011-06-20 14:27:24 +00:00 |
|
Miroslav Stampar
|
17fac6f67f
|
minor update
|
2011-06-20 13:53:39 +00:00 |
|
Miroslav Stampar
|
29314f425e
|
minor fix
|
2011-06-20 13:42:31 +00:00 |
|
Miroslav Stampar
|
d9015ed800
|
fix for a bug reported by krasn@deventum.com
|
2011-06-20 13:25:19 +00:00 |
|
Miroslav Stampar
|
f09340fc89
|
minor update
|
2011-06-20 12:40:14 +00:00 |
|
Miroslav Stampar
|
4d1fa5596b
|
added support for --scope in --crawl mode
|
2011-06-20 12:37:51 +00:00 |
|
Miroslav Stampar
|
42746cc706
|
bug fix
|
2011-06-20 12:18:46 +00:00 |
|
Miroslav Stampar
|
67fab9f2e2
|
putting this to info messages (user needs to know at this place why is it waiting)
|
2011-06-20 12:17:19 +00:00 |
|
Miroslav Stampar
|
b1426b5131
|
bug fix
|
2011-06-20 12:11:09 +00:00 |
|
Miroslav Stampar
|
cda39ca350
|
minor update
|
2011-06-20 11:46:23 +00:00 |
|
Miroslav Stampar
|
07e2c72943
|
adding Beautifulsoup (BSD) into extras; adding --crawl to options
|
2011-06-20 11:32:30 +00:00 |
|
Miroslav Stampar
|
8c04aa871a
|
english typo
|
2011-06-20 11:00:23 +00:00 |
|
Bernardo Damele
|
d7da71ce8e
|
politeness
|
2011-06-20 09:10:04 +00:00 |
|
Miroslav Stampar
|
bdb530da1f
|
minor update
|
2011-06-19 10:11:27 +00:00 |
|
Miroslav Stampar
|
d5bc149636
|
made changes by buawig request (504 is treated as a classical timeout)
|
2011-06-19 09:57:41 +00:00 |
|
Miroslav Stampar
|
83af83da9e
|
minor beautification (WordsSet is considered as a bad english)
|
2011-06-18 15:47:19 +00:00 |
|
Bernardo Damele
|
4b94ef2b7c
|
A little bit more polite
|
2011-06-18 13:03:55 +00:00 |
|
Bernardo Damele
|
f8c32cf6b9
|
Moved folder
|
2011-06-18 12:34:41 +00:00 |
|
Bernardo Damele
|
28ef61b997
|
Use getPageTextWordsSet() also in --common-columns
|
2011-06-18 12:30:26 +00:00 |
|
Bernardo Damele
|
6b2f44de14
|
Minor layout adjustment
|
2011-06-18 12:27:12 +00:00 |
|
Miroslav Stampar
|
ca6f9acf30
|
minor fix for resuming in multi threading mode
|
2011-06-18 12:23:18 +00:00 |
|
Bernardo Damele
|
cd07139919
|
Layout adjustments
|
2011-06-18 11:58:14 +00:00 |
|
Miroslav Stampar
|
31ad0875b4
|
added by request
|
2011-06-18 11:34:51 +00:00 |
|
Miroslav Stampar
|
e4be141602
|
minor fix for --smoke-test
|
2011-06-18 11:26:17 +00:00 |
|
Bernardo Damele
|
c7e1aeeef2
|
layout
|
2011-06-18 11:02:48 +00:00 |
|
Miroslav Stampar
|
905fef0eae
|
now user can explicitly state number of UNION affected columns via --union-cols (e.g. --union-cols=5)
|
2011-06-18 10:51:14 +00:00 |
|
Miroslav Stampar
|
7c537f6896
|
adding Chrome to the user-agents.txt
|
2011-06-18 10:12:06 +00:00 |
|
Miroslav Stampar
|
0c5d7d4535
|
removing crawling random agent strings as some sites appear different to them (minor possibility to screw blind engine)
|
2011-06-18 09:56:21 +00:00 |
|
Miroslav Stampar
|
fde3e4cece
|
better
|
2011-06-18 09:52:07 +00:00 |
|
Miroslav Stampar
|
2f129b01c0
|
"Please consider to provide" is a bad English
|
2011-06-18 09:46:22 +00:00 |
|
Miroslav Stampar
|
1440c9f2d4
|
minor update
|
2011-06-17 22:28:07 +00:00 |
|
Miroslav Stampar
|
87e9842371
|
better language
|
2011-06-17 22:13:45 +00:00 |
|
Miroslav Stampar
|
ce3170edef
|
minor update/better language
|
2011-06-17 22:11:40 +00:00 |
|
Miroslav Stampar
|
ec6fa384eb
|
update
|
2011-06-17 22:04:25 +00:00 |
|
Miroslav Stampar
|
0c9fa5c550
|
fix
|
2011-06-17 17:12:47 +00:00 |
|
Miroslav Stampar
|
043f2f92c1
|
minor update
|
2011-06-17 17:10:52 +00:00 |
|
Miroslav Stampar
|
c9a6aad5c3
|
minor fix by request
|
2011-06-17 16:58:50 +00:00 |
|
Miroslav Stampar
|
a0129dcbcb
|
this is confusing for normal users (i've just get a mail where dude thinks that he needs to use tamper script because of this :)
|
2011-06-17 16:52:39 +00:00 |
|
Miroslav Stampar
|
f3ee2c09fb
|
cleaner fix
|
2011-06-17 15:32:23 +00:00 |
|
Miroslav Stampar
|
bb987ec98f
|
fix for DNS leakage
|
2011-06-17 15:23:58 +00:00 |
|
Miroslav Stampar
|
9498a3f259
|
little stabilization of multi threading
|
2011-06-17 12:50:28 +00:00 |
|
Miroslav Stampar
|
2171c64213
|
update by request
|
2011-06-16 15:05:04 +00:00 |
|
Miroslav Stampar
|
d27afaed7e
|
some fixes
|
2011-06-16 14:27:44 +00:00 |
|
Miroslav Stampar
|
6b1d5a0ab8
|
minor fix
|
2011-06-16 14:11:30 +00:00 |
|
Miroslav Stampar
|
530c296519
|
minor fix
|
2011-06-16 13:56:17 +00:00 |
|