157 Commits

Author	SHA1	Message	Date
M09Ic	f1b9400e19	refactor: remove internal pkg, use engine replace ... fix: chunk mod not read fix: nil bar panic enhance: add default accept and user-agent	2025-02-22 20:31:32 +08:00
M09Ic	02162cffd6	revert: not same redirect banned	2024-11-01 12:27:31 +08:00
M09Ic	9e74a17096	fix: path join not expect	2024-11-01 12:25:53 +08:00
M09Ic	0ca5c02de7	enhance: skip not same host redirect	2024-10-30 16:11:05 +08:00
M09Ic	5cb9aa119d	fix: not same domain filtered	2024-10-30 15:57:32 +08:00
M09Ic	6bbc6141ac	enhance: add 404 default fuzzystatus, 429 waf status	2024-10-16 14:47:29 +08:00
M09Ic	af82ae43b9	enhance probe output	2024-10-14 02:20:39 +08:00
M09Ic	2de8822b01	fix %EXT% not work in plugin , https://github.com/chainreactors/spray/issues/63	2024-09-10 16:47:49 +08:00
M09Ic	106f007693	union load appendwords and dict	2024-08-26 02:22:35 +08:00
M09Ic	77a5e58a2a	add --print print all preset config	2024-08-26 01:47:39 +08:00
M09Ic	06bd9820e7	adapt spray_dict load	2024-08-26 01:32:11 +08:00
M09Ic	105c426396	refactor plugin	2024-08-26 01:20:03 +08:00
M09Ic	ebc74c1987	fix multi http parse bug	2024-08-06 03:46:30 +08:00
M09Ic	da7a3d221d	support multi fingers features	2024-07-17 01:40:30 +08:00
M09Ic	40a1f90601	support preset port config	2024-07-14 03:27:20 +08:00
M09Ic	04944cf829	adapt fingers	2024-07-01 19:51:16 +08:00
M09Ic	3e63812ab0	adapt all finger engine	2024-06-29 03:56:44 +08:00
M09Ic	cec35ec667	adapt new fingers engine and refactor output format	2024-06-29 03:49:29 +08:00
M09Ic	cbf60343bb	fix expr not work bug	2024-06-06 18:11:45 +08:00
M09Ic	b84535a06b	fix NewBaseline panic	2024-06-04 16:24:36 +08:00
M09ic	cbb8e2f295	misc update, bar and finger panic	2024-05-21 18:05:27 +08:00
M09ic	75680c21f4	update dependency	2024-05-16 18:35:39 +08:00
M09Ic	4e78e55b6e	fix config load priority, now config < cmd ... enhance bar print fix config.yaml default	2024-03-07 04:15:28 +08:00
M09Ic	9e82bb1ab3	use mpb replace uiprogress	2024-03-07 00:24:30 +08:00
M09Ic	16f149dadf	enhance crawl print	2024-03-06 18:26:59 +08:00
M09Ic	c70d26fd84	fix read body bug when MaxBodyLength == 0	2024-03-06 17:55:18 +08:00
M09Ic	d1d6982282	refactor fingers	2024-03-04 20:03:06 +08:00
M09Ic	9eb4a13e20	进一步优化日志输出, 减少非必要输出	2024-02-20 21:28:07 +08:00
M09Ic	88dc9fe7cd	fix fingerprinthub and fingers match bug	2024-02-20 21:09:00 +08:00
M09Ic	b9e970eb91	fix fingerpinthub match bug	2024-02-20 19:34:35 +08:00
M09Ic	376e8cdc38	issues19, add custom extract config	2024-02-20 19:12:05 +08:00
M09Ic	a40655287a	fix statistor	2024-02-20 18:52:43 +08:00
M09Ic	20993e5500	support fingerprinthub detect	2024-02-20 18:25:43 +08:00
M09Ic	61167054ee	refactor pool	2024-02-12 02:54:07 +08:00
M09Ic	c1309fe7da	修复functions重复生成目标的bug ... 优化自动识别能力, 404,200,403现在都会自动进行unique append-file 现在生效于uniquestatus与whitestatus	2024-02-10 12:48:22 +08:00
M09Ic	09c2a86a18	refactor package dependency	2024-02-08 15:26:01 +08:00
M09Ic	007ff96478	进一步简化日志, 提供-v参数开启详细日志	2024-02-08 14:57:33 +08:00
M09Ic	051c270df8	add dicc.txt as default dict	2024-02-07 02:59:55 +08:00
M09Ic	2222b49632	修复代理为空时的报错bug	2024-02-07 01:54:31 +08:00
M09Ic	f35a07050e	support http/socks5 proxy	2024-02-07 01:29:05 +08:00
M09Ic	ea5218b7f3	修复fasthttp client中, timeout不生效的bug	2024-02-07 00:46:11 +08:00
M09Ic	4cbb7b05ba	bump dependency	2023-12-28 14:34:19 +08:00
M09Ic	00c880f16e	优化爬虫的url处理	2023-06-30 17:43:47 +08:00
M09Ic	a36e06e96b	修复--timeout实际没生效的bug	2023-06-30 17:29:49 +08:00
M09Ic	73f724b92a	feat. 新增--index --random 支持自定义index与random目录	2023-06-03 22:24:52 +08:00
M09Ic	fc3f476fe2	修复bytes类型转换错误的bug	2023-06-03 21:09:01 +08:00
M09Ic	1a656b26a1	新增--retry插件	2023-05-04 12:04:59 +08:00
M09Ic	4b78503d64	添加url错误是的报错信息	2023-04-25 17:33:07 +08:00
M09Ic	0167f376db	修复派生的url可能导致部分插件出错的bug. 现在一旦url出现错误, 判定该请求无效	2023-04-25 17:28:08 +08:00
M09Ic	3698d01903	添加--scope与--no-scope参数, 用来指定爬虫的作用范围 ... 修复recursive中的多个bug	2023-04-14 20:05:21 +08:00