AI somewhat added
This commit is contained in:
@@ -0,0 +1,60 @@
|
||||
# Facebook Scraper - Configuration Needed
|
||||
|
||||
## Proxy Configuration
|
||||
|
||||
**File:** `rust-ai/src/main.rs`
|
||||
**Lines:** 25-27
|
||||
|
||||
The scraper needs real proxy URLs. The dummy placeholder `"http://0.0.0.0:0"` will fail to connect (expected — no crash, just error logs):
|
||||
|
||||
```rust
|
||||
const PROXIES: &[&str] = &[
|
||||
"http://0.0.0.0:0", // <- Replace with real proxy(es)
|
||||
];
|
||||
```
|
||||
|
||||
Replace with your actual proxies, e.g.:
|
||||
```rust
|
||||
const PROXIES: &[&str] = &[
|
||||
"http://user:pass@192.168.1.1:8080",
|
||||
"http://user:pass@192.168.1.2:8080",
|
||||
];
|
||||
```
|
||||
|
||||
## User Agents (Optional)
|
||||
|
||||
**File:** `rust-ai/src/main.rs`
|
||||
**Lines:** 32-36
|
||||
|
||||
Add more realistic user agents here if needed.
|
||||
|
||||
## Scraper Target URL
|
||||
|
||||
**File:** `rust-ai/src/main.rs`
|
||||
**Line:** 68
|
||||
|
||||
Currently fetches `https://www.facebook.com/messages`. Change if needed.
|
||||
|
||||
## Background Thread
|
||||
|
||||
**File:** `rust-ai/src/main.rs`
|
||||
**Lines:** 355-371
|
||||
|
||||
Runs in a `tokio::task::spawn_blocking` thread (changed from `tokio::spawn` because the scraper uses `reqwest::blocking::Client` + `thread::sleep`).
|
||||
|
||||
## Expected Behavior Until Configured
|
||||
|
||||
Until real proxies are set, the Rust server will log this every 1-5 seconds (not a crash):
|
||||
```
|
||||
ERROR crm_ai: Facebook scraper error: error sending request for url (https://www.facebook.com/messages)
|
||||
```
|
||||
Once you add working proxies, these errors stop and actual scraping begins.
|
||||
|
||||
## How it works
|
||||
|
||||
- Runs every 1-5 seconds on a background blocking thread (line 355-371)
|
||||
- Rotates through proxies and user agents for each request
|
||||
- Parses conversation HTML via `scraper` crate
|
||||
- Designed to detect job posts like "need a website" and notify sales reps
|
||||
- Error handling added to prevent server crash (uses `match` instead of `.unwrap()` at line 69)
|
||||
|
||||
Reference in New Issue
Block a user