61 lines
1.7 KiB
Markdown
61 lines
1.7 KiB
Markdown
# Facebook Scraper - Configuration Needed
|
|
|
|
## Proxy Configuration
|
|
|
|
**File:** `rust-ai/src/main.rs`
|
|
**Lines:** 25-27
|
|
|
|
The scraper needs real proxy URLs. The dummy placeholder `"http://0.0.0.0:0"` will fail to connect (expected — no crash, just error logs):
|
|
|
|
```rust
|
|
const PROXIES: &[&str] = &[
|
|
"http://0.0.0.0:0", // <- Replace with real proxy(es)
|
|
];
|
|
```
|
|
|
|
Replace with your actual proxies, e.g.:
|
|
```rust
|
|
const PROXIES: &[&str] = &[
|
|
"http://user:pass@192.168.1.1:8080",
|
|
"http://user:pass@192.168.1.2:8080",
|
|
];
|
|
```
|
|
|
|
## User Agents (Optional)
|
|
|
|
**File:** `rust-ai/src/main.rs`
|
|
**Lines:** 32-36
|
|
|
|
Add more realistic user agents here if needed.
|
|
|
|
## Scraper Target URL
|
|
|
|
**File:** `rust-ai/src/main.rs`
|
|
**Line:** 68
|
|
|
|
Currently fetches `https://www.facebook.com/search/top/?q=need%20website%20create`. Change if needed.
|
|
|
|
## Background Thread
|
|
|
|
**File:** `rust-ai/src/main.rs`
|
|
**Lines:** 355-371
|
|
|
|
Runs in a `tokio::task::spawn_blocking` thread (changed from `tokio::spawn` because the scraper uses `reqwest::blocking::Client` + `thread::sleep`).
|
|
|
|
## Expected Behavior Until Configured
|
|
|
|
Until real proxies are set, the Rust server will log this every 1-5 seconds (not a crash):
|
|
```
|
|
ERROR crm_ai: Facebook scraper error: error sending request for url (https://www.facebook.com/messages)
|
|
```
|
|
Once you add working proxies, these errors stop and actual scraping begins.
|
|
|
|
## How it works
|
|
|
|
- Runs every 1-5 seconds on a background blocking thread (line 355-371)
|
|
- Rotates through proxies and user agents for each request
|
|
- Parses conversation HTML via `scraper` crate
|
|
- Designed to detect job posts like "need a website" and notify sales reps
|
|
- Error handling added to prevent server crash (uses `match` instead of `.unwrap()` at line 69)
|
|
|