wiki:CrawlerExample

Version 1 (modified by wander, 11 years ago) (diff)

--

$ python crawler.py mil Got IP for EUR2.NIPR.mil.: 199.252.143.234 Got IP for PAC1.NIPR.mil.: 199.252.180.234 Got IP for PAC2.NIPR.mil.: 199.252.155.234 Got IP for EUR1.NIPR.mil.: 199.252.154.234 Got IP for CON2.NIPR.mil.: 199.252.162.234 Got IP for CON1.NIPR.mil.: 199.252.157.234 Connecting to database... ?Lucky hit: mil. ?Lucky hit: mil. Assuming closest encloser mil. -> 7GBLJ4F0H31BHS1JSPQ4T588RJ3OOHTS Assuming wildcard *.mil. -> PS4N9NTRKV7GKNODADGLIN99UVU539T5 paramset: ParamSet?[NSEC3, Algorithm=1, Flags=None, Iterations=10, Salt=ebfc paramid: 1794 Loading missing Gaps for paramid=1794 from database... No ranges in database yet Fresh run ?!!!Progress: 2.16 % Crawled 3 out of estimated 138 hashes 0 gaps in memory for mil. SQL COMMIT Testing whether old guesses cut current Gaps...

Hashed 0 old guesses, switching to regular mainloop #BUILD STATUS

?.|!.?.|!.,#?.|!.?.|!.#?.|!.?.|!.#?.|!.?.|!.#?.|!.?.<!.#?.|!.#?.|!.?.|!.?.|!.#?.|!.?.|!.#?.|!.?.<!.?.|!.#?.<!.#?.|!.#?.>!.#?.|!.?.|!.#?.|!.#?.|!.#?.|!.#?.>!.?.>!.#?.<!.?.|!.#?.|!.#?.|!.?.|!.#?.<!.#?.|!.#?ERROR: timeout at attempt=1 for server=199.252.154.234, retrying ?.>!.#?.>!.#?.>!.#?.|!.?.|!.#?.|!.?.|!.#?.>!.#?.>!.?.|!.?.>!.#?.|!.#?.|!.#?.|!.#?.<!.#?.|!.41 gaps in memory for mil. SQL COMMIT #?.|!.#?.|!.#?ERROR: timeout at attempt=1 for server=199.252.180.234, retrying ?.=!.#?.>!.?.|!.?.|!.?ERROR: timeout at attempt=1 for server=199.252.155.234, retrying ?.|!.#?.|!.#?.|!.#?.<!.?.>!.#?.|!.?.|!.?.|!.#?.|!.?.>!.?.<!.#?.|!.?.|!.#?.<!.?.|!.#?.>!.?.|!.?.=!.#?.|!.#?.=!.?.<!.#?.<!.#?.|!.?.>!.?.|!.#?.>!.?.|!.#?.|!.?.|!.59 gaps in memory for mil. SQL COMMIT #?.>!.?.>!.#?.|!.?.<!.?.>!.#?.|!.#?.<!.?.|!.?.|!.?.=!.#?.|!.#?.>!.?.>!.,#?.|!.?.|!.#?.|!.?.=!.#?.<!.?.>!.?.|!.,#?.>!.?.<!.?.>!.?.<!.#?.>!.?.|!.#?.=!.?.<!.?.|!.?.>!.#?.=!.?.<!.?.<!.#?.=!.?.>!.#?.>!.?.<!.?.=!.#?.=!.?.|!.?.>!.#?.=!.?.>!.#?.<!.?.>!.?.|!.#?ERROR: timeout at attempt=1 for server=199.252.162.234, retrying ?.|!.?.>!.?.=!.#?.<!.?.=!.?.=!.?.<!.#?.<!.?.=!.#?.|!.#?.=!.?.>!.61 gaps in memory for mil. SQL COMMIT #?.<!.?.=!.?.=!.#?.>!.,#?.>!.?.=!.?.|!.#?.=!.?.=!.?.=!.?.=!.#?.=!.?.=!.?.<!.#?.=!.?.=!.?ERROR: timeout at attempt=1 for server=199.252.157.234, retrying ?.=!.?.=!.#?.=!.?.=!.?.|!.#?.<!.?.=!.?.=!.#?.<!.?.=!.?.=!.#?.=!.?.=!.#?.>!.?.=!.?.|!.?.>!.#?.=!.?.=!.?.=!.,#?.>!.?.Found wraparound NSEC3 RR, reloading gaps from DB... !.?.=!.?.<!.#?.=!.?.=!.?.>!.?.=!.#?.<!.?.=!.?.>!.?.=!.#?.=!.?.<!.?.=!.?.=!.#?.<!.?.<!.?.=!.?.<!.#?.=!.?.=!.?.=!.?.=!.#?.=!.?.=!.?.=!.,#?.>!.?.=!.?.>!.?.=!.#?.=!.?.=!.?.|!.?.=!.#?.=!.,?.=!.?.=!.#?.=!.?.=!.#?.=!.?ERROR: timeout at attempt=1 for server=199.252.154.234, retrying ?.=!.?.=!.?.=!.7 gaps in memory for mil. SQL COMMIT #?.=!.#?.=!.?ERROR: timeout at attempt=1 for server=199.252.155.234, retrying ?.=!.,#?.=!.#?.=!.#?.=!.#?.=!.Progress: 100.0 % Crawled 235 out of estimated 235 hashes 0 gaps in memory for mil. SQL COMMIT