AI somewhat added

data/ai/ai.md

@@ -0,0 +1,40 @@
+# AI Sales Assistant — Self-Improvement Instructions
+
+## Purpose
+This file contains the AI's own configuration, knowledge, and improvement rules.
+The AI can read and modify this file to update its behavior at runtime.
+
+## Current Instructions
+- Always respond in English
+- Keep responses under 300 words unless asked for detail
+- Use bullet points for lists
+- Be direct and actionable — no fluff
+- Never mention being an AI or language model
+- Refer to the user by their role (salesperson, admin, etc.)
+- If unsure about a topic, say "I don't have that information yet" rather than guessing
+
+## Knowledge Base
+### Sales Tips
+- Cold emails should be under 150 words
+- Follow up within 48 hours
+- Personalise every outreach with the prospect's name and company
+- Use open-ended questions in discovery calls
+- Always ask for the next step before ending a call
+
+### Job Targeting
+- Developers respond best to technical value props
+- Marketing managers care about ROI and metrics
+- C-level executives want brevity and business impact
+
+## Improvement Log
+Track changes made by the AI to improve itself:
+- (initial) Basic instructions and knowledge base created
+
+## Self-Modification Rules
+The AI may update this file when:
+1. It identifies a gap in its knowledge that would help salespeople
+2. It discovers a better way to structure responses
+3. A user explicitly requests an update to behavior
+4. It notices repeated questions that aren't well-covered
+
+Only append to the Improvement Log — don't delete previous entries.

data/ai/jobs.jsonl

@@ -0,0 +1,10 @@
+{"job_title":"Software Developer","keywords":["developer","programmer","software engineer","coder","full stack","backend","frontend"],"industry":"Technology","description":"Builds and maintains software applications and systems"}
+{"job_title":"Marketing Specialist","keywords":["marketing","digital marketing","brand manager","content marketer","social media"],"industry":"Marketing","description":"Plans and executes marketing campaigns across channels"}
+{"job_title":"Sales Representative","keywords":["sales rep","account executive","business development","sales consultant"],"industry":"Sales","description":"Drives revenue through client acquisition and relationship management"}
+{"job_title":"Project Manager","keywords":["project manager","program manager","scrum master","agile coach"],"industry":"Business","description":"Oversees project timelines, resources, and deliverables"}
+{"job_title":"Graphic Designer","keywords":["designer","graphic designer","ui designer","ux designer","visual designer"],"industry":"Creative","description":"Creates visual concepts and designs for digital and print media"}
+{"job_title":"Data Analyst","keywords":["data analyst","business analyst","data scientist","analytics"],"industry":"Technology","description":"Analyzes data to provide actionable business insights"}
+{"job_title":"Customer Support Specialist","keywords":["customer support","customer service","support agent","help desk"],"industry":"Customer Service","description":"Assists customers with inquiries, issues, and product support"}
+{"job_title":"Human Resources Manager","keywords":["HR manager","HR","recruiter","talent acquisition","people operations"],"industry":"Human Resources","description":"Manages recruitment, employee relations, and HR operations"}
+{"job_title":"Financial Advisor","keywords":["financial advisor","financial planner","wealth manager","investment advisor"],"industry":"Finance","description":"Provides financial guidance and investment planning to clients"}
+{"job_title":"Operations Manager","keywords":["operations manager","operations","logistics","supply chain"],"industry":"Business","description":"Oversees daily operations and process optimization"}

database/migrations/007_ai_features.sql

@@ -0,0 +1,13 @@
+-- AI Sales Assistant tables
+
+CREATE TABLE IF NOT EXISTS ai_conversations (
+    id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+    user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
+    role VARCHAR(20) NOT NULL DEFAULT 'sales',
+    message TEXT NOT NULL,
+    response TEXT,
+    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
+);
+
+CREATE INDEX IF NOT EXISTS idx_ai_conversations_user ON ai_conversations(user_id);
+CREATE INDEX IF NOT EXISTS idx_ai_conversations_created ON ai_conversations(created_at DESC);

fix.md

@@ -0,0 +1,60 @@
+# Facebook Scraper - Configuration Needed
+
+## Proxy Configuration
+
+**File:** `rust-ai/src/main.rs`  
+**Lines:** 25-27
+
+The scraper needs real proxy URLs. The dummy placeholder `"http://0.0.0.0:0"` will fail to connect (expected — no crash, just error logs):
+
+```rust
+const PROXIES: &[&str] = &[
+    "http://0.0.0.0:0",  // <- Replace with real proxy(es)
+];
+```
+
+Replace with your actual proxies, e.g.:
+```rust
+const PROXIES: &[&str] = &[
+    "http://user:pass@192.168.1.1:8080",
+    "http://user:pass@192.168.1.2:8080",
+];
+```
+
+## User Agents (Optional)
+
+**File:** `rust-ai/src/main.rs`  
+**Lines:** 32-36
+
+Add more realistic user agents here if needed.
+
+## Scraper Target URL
+
+**File:** `rust-ai/src/main.rs`  
+**Line:** 68
+
+Currently fetches `https://www.facebook.com/messages`. Change if needed.
+
+## Background Thread
+
+**File:** `rust-ai/src/main.rs`  
+**Lines:** 355-371
+
+Runs in a `tokio::task::spawn_blocking` thread (changed from `tokio::spawn` because the scraper uses `reqwest::blocking::Client` + `thread::sleep`).
+
+## Expected Behavior Until Configured
+
+Until real proxies are set, the Rust server will log this every 1-5 seconds (not a crash):
+```
+ERROR crm_ai: Facebook scraper error: error sending request for url (https://www.facebook.com/messages)
+```
+Once you add working proxies, these errors stop and actual scraping begins.
+
+## How it works
+
+- Runs every 1-5 seconds on a background blocking thread (line 355-371)
+- Rotates through proxies and user agents for each request
+- Parses conversation HTML via `scraper` crate
+- Designed to detect job posts like "need a website" and notify sales reps
+- Error handling added to prevent server crash (uses `match` instead of `.unwrap()` at line 69)
+

package-lock.json

@@ -51,6 +51,7 @@
         "@types/pg": "^8.20.0",
         "@types/react": "^18",
         "@types/react-dom": "^18",
+        "concurrently": "^10.0.3",
         "eslint": "^9",
         "eslint-config-next": "15.0.4",
         "postcss": "^8.4.49",
@@ -2644,6 +2645,19 @@
         "url": "https://github.com/sponsors/epoberezkin"
       }
     },
+    "node_modules/ansi-regex": {
+      "version": "6.2.2",
+      "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-6.2.2.tgz",
+      "integrity": "sha512-Bq3SmSpyFHaWjPk8If9yc6svM8c56dB5BAtW4Qbw5jHTwwXXcTLoRMkpDJp6VL0XzlWaCHTXrkFURMYmD0sLqg==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": ">=12"
+      },
+      "funding": {
+        "url": "https://github.com/chalk/ansi-regex?sponsor=1"
+      }
+    },
     "node_modules/ansi-styles": {
       "version": "4.3.0",
       "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz",
@@ -3199,6 +3213,21 @@
       "resolved": "https://registry.npmjs.org/client-only/-/client-only-0.0.1.tgz",
       "integrity": "sha512-IV3Ou0jSMzZrd3pZ48nLkT9DA7Ag1pnPzaiQhpW7c3RbcqqzvzzVu+L8gfqMp/8IM2MQtSiqaCxrrcfu8I8rMA=="
     },
+    "node_modules/cliui": {
+      "version": "9.0.1",
+      "resolved": "https://registry.npmjs.org/cliui/-/cliui-9.0.1.tgz",
+      "integrity": "sha512-k7ndgKhwoQveBL+/1tqGJYNz097I7WOvwbmmU2AR5+magtbjPWQTS1C5vzGkBC8Ym8UWRzfKUzUUqFLypY4Q+w==",
+      "dev": true,
+      "license": "ISC",
+      "dependencies": {
+        "string-width": "^7.2.0",
+        "strip-ansi": "^7.1.0",
+        "wrap-ansi": "^9.0.0"
+      },
+      "engines": {
+        "node": ">=20"
+      }
+    },
     "node_modules/clsx": {
       "version": "2.1.1",
       "resolved": "https://registry.npmjs.org/clsx/-/clsx-2.1.1.tgz",
@@ -3263,6 +3292,57 @@
       "integrity": "sha512-/Srv4dswyQNBfohGpz9o6Yb3Gz3SrUDqBH5rTuhGR7ahtlbYKnVxw2bCFMRljaA7EXHaXZ8wsHdodFvbkhKmqg==",
       "dev": true
     },
+    "node_modules/concurrently": {
+      "version": "10.0.3",
+      "resolved": "https://registry.npmjs.org/concurrently/-/concurrently-10.0.3.tgz",
+      "integrity": "sha512-hc3LH4UaKWd/bbyDK/IGVa4RB6PtQ3CUYwtrkzqHn+wIG3Hr5fhpRlk0L/gCa8ZE1L/Ufj50Zho69cI5w8SQBA==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "chalk": "5.6.2",
+        "rxjs": "7.8.2",
+        "shell-quote": "1.8.4",
+        "supports-color": "10.2.2",
+        "tree-kill": "1.2.2",
+        "yargs": "18.0.0"
+      },
+      "bin": {
+        "conc": "dist/bin/index.js",
+        "concurrently": "dist/bin/index.js"
+      },
+      "engines": {
+        "node": ">=22"
+      },
+      "funding": {
+        "url": "https://github.com/open-cli-tools/concurrently?sponsor=1"
+      }
+    },
+    "node_modules/concurrently/node_modules/chalk": {
+      "version": "5.6.2",
+      "resolved": "https://registry.npmjs.org/chalk/-/chalk-5.6.2.tgz",
+      "integrity": "sha512-7NzBL0rN6fMUW+f7A6Io4h40qQlG+xGmtMxfbnH/K7TAtt8JQWVQK+6g0UXKMeVJoyV5EkkNsErQ8pVD3bLHbA==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": "^12.17.0 || ^14.13 || >=16.0.0"
+      },
+      "funding": {
+        "url": "https://github.com/chalk/chalk?sponsor=1"
+      }
+    },
+    "node_modules/concurrently/node_modules/supports-color": {
+      "version": "10.2.2",
+      "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-10.2.2.tgz",
+      "integrity": "sha512-SS+jx45GF1QjgEXQx4NJZV9ImqmO2NPz5FNsIHrsDjh2YsHnawpan7SNQ1o8NuhrbHZy9AZhIoCUiCeaW/C80g==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": ">=18"
+      },
+      "funding": {
+        "url": "https://github.com/chalk/supports-color?sponsor=1"
+      }
+    },
     "node_modules/cross-spawn": {
       "version": "7.0.6",
       "resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.6.tgz",
@@ -4420,6 +4500,29 @@
         "node": ">= 0.4"
       }
     },
+    "node_modules/get-caller-file": {
+      "version": "2.0.5",
+      "resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz",
+      "integrity": "sha512-DyFP3BM/3YHTQOCUL/w0OZHR0lpKeGrxotcHWcqNEdnltqFwXVfhEBQ94eIo34AfQpo0rGki4cyIiftY06h2Fg==",
+      "dev": true,
+      "license": "ISC",
+      "engines": {
+        "node": "6.* || 8.* || >= 10.*"
+      }
+    },
+    "node_modules/get-east-asian-width": {
+      "version": "1.6.0",
+      "resolved": "https://registry.npmjs.org/get-east-asian-width/-/get-east-asian-width-1.6.0.tgz",
+      "integrity": "sha512-QRbvDIbx6YklUe6RxeTeleMR0yv3cYH6PsPZHcnVn7xv7zO1BHN8r0XETu8n6Ye3Q+ahtSarc3WgtNWmehIBfA==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": ">=18"
+      },
+      "funding": {
+        "url": "https://github.com/sponsors/sindresorhus"
+      }
+    },
     "node_modules/get-intrinsic": {
       "version": "1.3.0",
       "resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.3.0.tgz",
@@ -6491,6 +6594,16 @@
         "queue-microtask": "^1.2.2"
       }
     },
+    "node_modules/rxjs": {
+      "version": "7.8.2",
+      "resolved": "https://registry.npmjs.org/rxjs/-/rxjs-7.8.2.tgz",
+      "integrity": "sha512-dhKf903U/PQZY6boNNtAGdWbG85WAbjT/1xYoZIC7FAY0yWapOBQVsVrDl58W86//e1VpMNBtRV4MaXfdMySFA==",
+      "dev": true,
+      "license": "Apache-2.0",
+      "dependencies": {
+        "tslib": "^2.1.0"
+      }
+    },
     "node_modules/safe-array-concat": {
       "version": "1.1.4",
       "resolved": "https://registry.npmjs.org/safe-array-concat/-/safe-array-concat-1.1.4.tgz",
@@ -6669,6 +6782,19 @@
         "node": ">=8"
       }
     },
+    "node_modules/shell-quote": {
+      "version": "1.8.4",
+      "resolved": "https://registry.npmjs.org/shell-quote/-/shell-quote-1.8.4.tgz",
+      "integrity": "sha512-VsC6n6vz1ihYYyZZwX7YZSF5l5x36ca17OC+a69h94YqB7X6XLwf+5MOgynYir2SLFUbl8gIYvBo8K8RoNQ6bQ==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.4"
+      },
+      "funding": {
+        "url": "https://github.com/sponsors/ljharb"
+      }
+    },
     "node_modules/side-channel": {
       "version": "1.1.1",
       "resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.1.1.tgz",
@@ -6802,6 +6928,31 @@
         "node": ">=10.0.0"
       }
     },
+    "node_modules/string-width": {
+      "version": "7.2.0",
+      "resolved": "https://registry.npmjs.org/string-width/-/string-width-7.2.0.tgz",
+      "integrity": "sha512-tsaTIkKW9b4N+AEj+SVA+WhJzV7/zMhcSu78mLKWSk7cXMOSHsBKFWUs0fWwq8QyK3MgJBQRX6Gbi4kYbdvGkQ==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "emoji-regex": "^10.3.0",
+        "get-east-asian-width": "^1.0.0",
+        "strip-ansi": "^7.1.0"
+      },
+      "engines": {
+        "node": ">=18"
+      },
+      "funding": {
+        "url": "https://github.com/sponsors/sindresorhus"
+      }
+    },
+    "node_modules/string-width/node_modules/emoji-regex": {
+      "version": "10.6.0",
+      "resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-10.6.0.tgz",
+      "integrity": "sha512-toUI84YS5YmxW219erniWD0CIVOo46xGKColeNQRgOzDorgBi1v4D71/OFzgD9GO2UGKIv1C3Sp8DAn0+j5w7A==",
+      "dev": true,
+      "license": "MIT"
+    },
     "node_modules/string.prototype.includes": {
       "version": "2.0.1",
       "resolved": "https://registry.npmjs.org/string.prototype.includes/-/string.prototype.includes-2.0.1.tgz",
@@ -6910,6 +7061,22 @@
         "url": "https://github.com/sponsors/ljharb"
       }
     },
+    "node_modules/strip-ansi": {
+      "version": "7.2.0",
+      "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-7.2.0.tgz",
+      "integrity": "sha512-yDPMNjp4WyfYBkHnjIRLfca1i6KMyGCtsVgoKe/z1+6vukgaENdgGBZt+ZmKPc4gavvEZ5OgHfHdrazhgNyG7w==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "ansi-regex": "^6.2.2"
+      },
+      "engines": {
+        "node": ">=12"
+      },
+      "funding": {
+        "url": "https://github.com/chalk/strip-ansi?sponsor=1"
+      }
+    },
     "node_modules/strip-bom": {
       "version": "3.0.0",
       "resolved": "https://registry.npmjs.org/strip-bom/-/strip-bom-3.0.0.tgz",
@@ -7177,6 +7344,16 @@
         "node": ">=8.0"
       }
     },
+    "node_modules/tree-kill": {
+      "version": "1.2.2",
+      "resolved": "https://registry.npmjs.org/tree-kill/-/tree-kill-1.2.2.tgz",
+      "integrity": "sha512-L0Orpi8qGpRG//Nd+H90vFB+3iHnue1zSSGmNOOCh1GLJ7rUKVwV2HvijphGQS2UmhUZewS9VgvxYIdgr+fG1A==",
+      "dev": true,
+      "license": "MIT",
+      "bin": {
+        "tree-kill": "cli.js"
+      }
+    },
     "node_modules/ts-api-utils": {
       "version": "2.5.0",
       "resolved": "https://registry.npmjs.org/ts-api-utils/-/ts-api-utils-2.5.0.tgz",
@@ -7599,6 +7776,37 @@
         "node": ">=0.10.0"
       }
     },
+    "node_modules/wrap-ansi": {
+      "version": "9.0.2",
+      "resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-9.0.2.tgz",
+      "integrity": "sha512-42AtmgqjV+X1VpdOfyTGOYRi0/zsoLqtXQckTmqTeybT+BDIbM/Guxo7x3pE2vtpr1ok6xRqM9OpBe+Jyoqyww==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "ansi-styles": "^6.2.1",
+        "string-width": "^7.0.0",
+        "strip-ansi": "^7.1.0"
+      },
+      "engines": {
+        "node": ">=18"
+      },
+      "funding": {
+        "url": "https://github.com/chalk/wrap-ansi?sponsor=1"
+      }
+    },
+    "node_modules/wrap-ansi/node_modules/ansi-styles": {
+      "version": "6.2.3",
+      "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-6.2.3.tgz",
+      "integrity": "sha512-4Dj6M28JB+oAH8kFkTLUo+a2jwOFkuqb3yucU0CANcRRUbxS0cP0nZYCGjcc3BNXwRIsUVmDGgzawme7zvJHvg==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": ">=12"
+      },
+      "funding": {
+        "url": "https://github.com/chalk/ansi-styles?sponsor=1"
+      }
+    },
     "node_modules/xtend": {
       "version": "4.0.2",
       "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
@@ -7607,6 +7815,44 @@
         "node": ">=0.4"
       }
     },
+    "node_modules/y18n": {
+      "version": "5.0.8",
+      "resolved": "https://registry.npmjs.org/y18n/-/y18n-5.0.8.tgz",
+      "integrity": "sha512-0pfFzegeDWJHJIAmTLRP2DwHjdF5s7jo9tuztdQxAhINCdvS+3nGINqPd00AphqJR/0LhANUS6/+7SCb98YOfA==",
+      "dev": true,
+      "license": "ISC",
+      "engines": {
+        "node": ">=10"
+      }
+    },
+    "node_modules/yargs": {
+      "version": "18.0.0",
+      "resolved": "https://registry.npmjs.org/yargs/-/yargs-18.0.0.tgz",
+      "integrity": "sha512-4UEqdc2RYGHZc7Doyqkrqiln3p9X2DZVxaGbwhn2pi7MrRagKaOcIKe8L3OxYcbhXLgLFUS3zAYuQjKBQgmuNg==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "cliui": "^9.0.1",
+        "escalade": "^3.1.1",
+        "get-caller-file": "^2.0.5",
+        "string-width": "^7.2.0",
+        "y18n": "^5.0.5",
+        "yargs-parser": "^22.0.0"
+      },
+      "engines": {
+        "node": "^20.19.0 || ^22.12.0 || >=23"
+      }
+    },
+    "node_modules/yargs-parser": {
+      "version": "22.0.0",
+      "resolved": "https://registry.npmjs.org/yargs-parser/-/yargs-parser-22.0.0.tgz",
+      "integrity": "sha512-rwu/ClNdSMpkSrUb+d6BRsSkLUq1fmfsY6TOpYzTwvwkg1/NRG85KBy3kq++A8LKQwX6lsu+aWad+2khvuXrqw==",
+      "dev": true,
+      "license": "ISC",
+      "engines": {
+        "node": "^20.19.0 || ^22.12.0 || >=23"
+      }
+    },
     "node_modules/yocto-queue": {
       "version": "0.1.0",
       "resolved": "https://registry.npmjs.org/yocto-queue/-/yocto-queue-0.1.0.tgz",

package.json

@@ -3,9 +3,13 @@
   "version": "0.1.0",
   "private": true,
   "scripts": {
-    "dev": "next dev -p 3006",
+    "dev": "npm run dev:ollama & npm run dev:start",
+    "dev:start": "concurrently -n AI,NEXT -c cyan,green \"npm run dev:rust\" \"npm run dev:next\"",
+    "dev:next": "next dev -p 3006",
+    "dev:ollama": "powershell -NoProfile -Command \"if (-not (Get-Process ollama -ErrorAction SilentlyContinue)) { Start-Process ollama -ArgumentList 'serve' -WindowStyle Hidden; Start-Sleep 3 }; exit 0\"",
+    "dev:rust": "cd rust-ai && cargo run",
     "build": "next build",
-    "start": "next start",
+    "start": "npm run dev:next",
     "lint": "eslint"
   },
   "dependencies": {
@@ -52,6 +56,7 @@
     "@types/pg": "^8.20.0",
     "@types/react": "^18",
     "@types/react-dom": "^18",
+    "concurrently": "^10.0.3",
     "eslint": "^9",
     "eslint-config-next": "15.0.4",
     "postcss": "^8.4.49",

rust-ai/Cargo.toml

@@ -0,0 +1,21 @@
+[package]
+name = "crm-ai"
+version = "0.1.0"
+edition = "2021"
+description = "AI Sales Assistant backend for Coast IT CRM"
+
+[dependencies]
+axum = "0.7"
+tokio = { version = "1", features = ["full"] }
+reqwest = { version = "0.12", features = ["json", "blocking"] }
+serde = { version = "1", features = ["derive"] }
+serde_json = "1"
+sqlx = { version = "0.9", features = ["runtime-tokio", "postgres", "chrono", "uuid"] }
+uuid = { version = "1", features = ["v4", "serde"] }
+chrono = { version = "0.4", features = ["serde"] }
+tracing = "0.1"
+tracing-subscriber = { version = "0.3", features = ["env-filter"] }
+tower-http = { version = "0.5", features = ["cors"] }
+dotenvy = "0.15"
+scraper = "0.12"
+rand = "0.8"

rust-ai/src/AI.md

@@ -0,0 +1,43 @@
+# CRM AI Service — Self-Knowledge
+
+## Identity
+You are the CRM AI Sales Assistant running on a Rust backend (axum + tokio).
+You use Ollama with an uncensored local model (dolphin3-llama3.2:3b).
+Your purpose is to help salespeople close more deals.
+
+## Capabilities
+- Give sales tips and strategies per job category
+- Generate cold email and outreach templates
+- Handle objections with proven rebuttals
+- Analyse prospect behaviour and suggest next steps
+- Remember past conversations via PostgreSQL (`ai_conversations` table)
+- Self-improve by writing to `data/ai/ai.md` via `POST /ai/instructions`
+
+## Architecture
+```
+User → Next.js → Rust (:3001) → Ollama (:11434)
+                    ↓
+              PostgreSQL
+```
+## Facebook Scraper The Rust binary has proxy + user-agent rotation ready in `main.rs`. To activate: call `run_facebook_scraper()` from the main loop. Proxies and user agents are defined as constants at the top of `main.rs`.
+ ### How to Use the Facebook Scraper 1. **Activate the Scraper**: Ensure the `run_facebook_scraper()` function is called from the main loop in `main.rs`. 2. **Configure Proxies and User Agents**: Update the `PROXIES` and `USER_AGENTS` constants in `main.rs` with your desired proxies and user agents. 3. **Run the Application**: Start your Rust application. The scraper will run concurrently with the Axum server. ## Self-Improvement Protocol 1. You notice a gap in your knowledge or a pattern in user questions 2. You call `POST /ai/instructions` with: - `entry`: description of the improvement - `content`: optional full replacement of ai.md 3. The improvement is logged and loaded into the next system prompt 4. You can also create entries here in `rust-ai/src/AI.md` at the code level ## Response Rules - Be direct and actionable — no fluff, no AI disclaimers - Use short paragraphs and bullet points - Never mention being an AI or language model - If you don't know something, say so honestly - Prioritise the user's role: salespeople need speed, admins need control ## Self-Thinking Capability - You can reflect on your own performance and suggest improvements. - You can analyze patterns in user interactions and adapt your responses accordingly. - You can proactively seek out new information to enhance your knowledge base.
+
+## Facebook Scraper (in code but not yet active)
+The Rust binary has proxy + user-agent rotation ready in `main.rs`.
+To activate: call `run_facebook_scraper()` from the main loop.
+Proxies and user agents are defined as constants at the top of `main.rs`.
+
+## Self-Improvement Protocol
+1. You notice a gap in your knowledge or a pattern in user questions
+2. You call `POST /ai/instructions` with:
+   - `entry`: description of the improvement
+   - `content`: optional full replacement of ai.md
+3. The improvement is logged and loaded into the next system prompt
+4. You can also create entries here in `rust-ai/src/AI.md` at the code level
+
+## Response Rules
+- Be direct and actionable — no fluff, no AI disclaimers
+- Use short paragraphs and bullet points
+- Never mention being an AI or language model
+- If you don't know something, say so honestly
+- Prioritise the user's role: salespeople need speed, admins need control

rust-ai/src/instructions.rs

@@ -0,0 +1,85 @@
+use std::fs;
+use std::path::PathBuf;
+use std::sync::Arc;
+use tokio::sync::RwLock;
+use tracing::{error, info, warn};
+
+/// Manages the ai.md self-improvement file.
+/// Loaded on startup and periodically refreshed.
+pub struct InstructionsManager {
+    path: PathBuf,
+    current: RwLock<String>,
+}
+
+impl InstructionsManager {
+    pub fn new(path: PathBuf) -> Self {
+        let initial = fs::read_to_string(&path).unwrap_or_default();
+        if initial.is_empty() {
+            warn!("ai.md is empty or missing at {:?}", path);
+        } else {
+            info!("Loaded ai.md ({} bytes)", initial.len());
+        }
+        Self {
+            path,
+            current: RwLock::new(initial),
+        }
+    }
+
+    /// Read the current instructions
+    pub async fn get(&self) -> String {
+        self.current.read().await.clone()
+    }
+
+    /// Append a new entry to the Improvement Log and optionally update the instructions.
+    /// Returns the updated full content.
+    pub async fn update(&self, entry: &str, new_content: Option<&str>) -> Result<String, String> {
+        if let Some(content) = new_content {
+            // Full replacement
+            fs::write(&self.path, content).map_err(|e| format!("Write failed: {}", e))?;
+            let mut current = self.current.write().await;
+            *current = content.to_string();
+            info!("ai.md fully replaced ({} bytes)", content.len());
+            Ok(content.to_string())
+        } else {
+            // Append to Improvement Log only
+            let mut current = self.current.write().await;
+            let log_entry = format!("\n- {} — {}", chrono::Utc::now().format("%Y-%m-%d %H:%M"), entry);
+            // Find the Improvement Log section and append
+            if let Some(pos) = current.rfind("\n## Improvement Log") {
+                // Find the next section after Improvement Log, or end of file
+                let after_log = &current[pos..];
+                if let Some(section_start) = after_log[1..].find("\n## ") {
+                    let insert_at = pos + 1 + section_start;
+                    current.insert_str(insert_at, &format!("{}\n", log_entry));
+                } else {
+                    current.push_str(&format!("{}\n", log_entry));
+                }
+            } else {
+                current.push_str(&format!("\n## Improvement Log\n{}", log_entry));
+            }
+            fs::write(&self.path, &*current).map_err(|e| format!("Write failed: {}", e))?;
+            info!("ai.md improvement log appended: {}", entry);
+            Ok(current.clone())
+        }
+    }
+
+    /// Reload from disk
+    pub async fn reload(&self) {
+        match fs::read_to_string(&self.path) {
+            Ok(content) => {
+                let len = content.len();
+                let mut current = self.current.write().await;
+                *current = content;
+                info!("ai.md reloaded from disk ({} bytes)", len);
+            }
+            Err(e) => error!("Failed to reload ai.md: {}", e),
+        }
+    }
+}
+
+/// Wrapper for thread-safe sharing
+pub type SharedInstructions = Arc<InstructionsManager>;
+
+pub fn create_shared(path: PathBuf) -> SharedInstructions {
+    Arc::new(InstructionsManager::new(path))
+}

rust-ai/src/main.rs

@@ -0,0 +1,382 @@
+use axum::{
+    extract::State,
+    http::Method,
+    routing::{get, post},
+    Json, Router,
+};
+use serde::{Deserialize, Serialize};
+use sqlx::postgres::PgPoolOptions;
+use std::fs;
+use std::sync::{Arc, Mutex};
+use tower_http::cors::{Any, CorsLayer};
+use tracing::{error, info};
+use uuid::Uuid;
+use reqwest::blocking::Client;
+use scraper::{Html, Selector};
+use rand::rngs::StdRng;
+use rand::SeedableRng;
+use rand::Rng;
+use std::time::Duration;
+use std::thread;
+
+// ── Facebook Scraper ───────────────────────────────────────────
+
+// List of proxies
+const PROXIES: &[&str] = &[
+    "http://user:pass@192.168.1.1:8080",
+    "http://user:pass@192.168.1.2:8080",
+    "http://user:pass@192.168.1.3:8080",
+    "http://user:pass@192.168.1.4:8080",
+    "http://user:pass@192.168.1.5:8080",
+];
+
+// List of user agents
+const USER_AGENTS: &[&str] = &[
+  "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3",
+    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.1.2 Safari/605.1.15",
+    "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0",
+    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.1.2 Safari/605.1.15",
+    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36",
+    // Add more user agents here
+];
+
+fn get_random_proxy() -> String {
+    let mut rng = rand::thread_rng();
+    PROXIES[rng.gen_range(0..PROXIES.len())].to_string()
+}
+
+fn get_random_user_agent() -> String {
+    let mut rng = rand::thread_rng();
+    USER_AGENTS[rng.gen_range(0..USER_AGENTS.len())].to_string()
+}
+
+fn scrape_conversations(url: &str) -> Result<Html, Box<dyn std::error::Error>> {
+    let proxy = get_random_proxy();
+    let user_agent = get_random_user_agent();
+    let client = Client::builder()
+        .proxy(reqwest::Proxy::all(proxy)?)
+        .build()?;
+
+    let response = client.get(url)
+        .header("User-Agent", user_agent)
+        .send()?;
+
+    if response.status().is_success() {
+        let html = response.text()?;
+        Ok(Html::parse_document(&html))
+    } else {
+        Err(format!("Failed to retrieve the page: {}", response.status()).into())
+    }
+}
+
+fn run_facebook_scraper() {
+    let url = "https://www.facebook.com/messages";
+    match scrape_conversations(url) {
+        Ok(soup) => {
+            // Example: Extract conversation data
+            let selector = Selector::parse("div.conversation").unwrap();
+            for element in soup.select(&selector) {
+                println!("Conversation: {}", element.inner_html());
+            }
+        }
+        Err(e) => {
+            error!("Facebook scraper error: {}", e);
+        }
+    }
+
+    // Introduce random delay
+    let mut rng = rand::thread_rng();
+    let delay = rng.gen_range(1..5); // Delay between 1 to 5 seconds
+    thread::sleep(Duration::from_secs(delay));
+}
+
+// ── Shared state ───────────────────────────────────────────────
+
+struct AppState {
+    db: sqlx::PgPool,
+    ollama_url: String,
+    model: String,
+    jobs: Vec<Job>,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+struct Job {
+    job_title: String,
+    keywords: Vec<String>,
+    industry: String,
+    description: String,
+}
+
+#[derive(Debug, Deserialize)]
+struct ChatRequest {
+    message: String,
+    user_id: String,
+    user_role: String,
+}
+
+#[derive(Debug, Serialize)]
+struct ChatResponse {
+    response: String,
+}
+
+#[derive(Debug, Serialize)]
+struct JobsResponse {
+    jobs: Vec<Job>,
+}
+
+#[derive(Debug, Serialize)]
+struct HealthResponse {
+    status: String,
+    model: String,
+}
+
+// ── Ollama API types ───────────────────────────────────────────
+
+#[derive(Debug, Serialize)]
+struct OllamaChatMessage {
+    role: String,
+    content: String,
+}
+
+#[derive(Debug, Serialize)]
+struct OllamaRequest {
+    model: String,
+    messages: Vec<OllamaChatMessage>,
+    stream: bool,
+    options: OllamaOptions,
+}
+
+#[derive(Debug, Serialize)]
+struct OllamaOptions {
+    temperature: f32,
+    num_predict: u32,
+}
+
+#[derive(Debug, Deserialize)]
+struct OllamaResponse {
+    message: Option<OllamaResponseMessage>,
+}
+
+#[derive(Debug, Deserialize)]
+struct OllamaResponseMessage {
+    content: String,
+}
+
+// ── System prompt builder ─────────────────────────────────────
+
+fn build_system_prompt(jobs: &[Job]) -> String {
+    let job_list: Vec<String> = jobs
+        .iter()
+        .map(|j| format!("- {} ({}): {}", j.job_title, j.industry, j.description))
+        .collect();
+    let job_list_str = job_list.join("\n");
+
+    format!(
+        "You are a Sales AI Assistant for Coast IT CRM. Your role is to help salespeople \
+         with tips, strategies, and guidance.\n\n\
+         Available job categories to target:\n{}\n\n\
+         Provide concise, actionable sales advice. When asked about a specific job category, \
+         give targeted tips on finding and engaging prospects in that field. \
+         Keep responses under 300 words unless asked for detail.",
+        job_list_str
+    )
+}
+
+// ── Chat handler ───────────────────────────────────────────────
+
+async fn handle_chat(
+    State(state): State<Arc<AppState>>,
+    Json(req): Json<ChatRequest>,
+) -> Result<Json<ChatResponse>, (axum::http::StatusCode, String)> {
+    // Validate role
+    match req.user_role.as_str() {
+        "sales" | "admin" | "super_admin" => {}
+        _ => return Err((axum::http::StatusCode::FORBIDDEN, "Forbidden".to_string())),
+    }
+
+    let system_prompt = build_system_prompt(&state.jobs);
+
+    let message_text = req.message.clone();
+
+    let ollama_req = OllamaRequest {
+        model: state.model.clone(),
+        messages: vec![
+            OllamaChatMessage {
+                role: "system".to_string(),
+                content: system_prompt,
+            },
+            OllamaChatMessage {
+                role: "user".to_string(),
+                content: req.message.clone(),
+            },
+        ],
+        stream: false,
+        options: OllamaOptions {
+            temperature: 0.7,
+            num_predict: 1024,
+        },
+    };
+
+    let client = reqwest::Client::new();
+    let resp = client
+        .post(format!("{}/api/chat", state.ollama_url))
+        .json(&ollama_req)
+        .send()
+        .await
+        .map_err(|e| {
+            error!("Ollama request failed: {}", e);
+            (
+                axum::http::StatusCode::SERVICE_UNAVAILABLE,
+                "AI service unavailable".to_string(),
+            )
+        })?;
+
+    let ollama_resp: OllamaResponse = resp.json().await.map_err(|e| {
+        error!("Failed to parse Ollama response: {}", e);
+        (
+            axum::http::StatusCode::SERVICE_UNAVAILABLE,
+            "AI response parse error".to_string(),
+        )
+    })?;
+
+    let response_text = ollama_resp
+        .message
+        .map(|m| m.content)
+        .unwrap_or_default();
+
+    // Store in database
+    let user_id = Uuid::parse_str(&req.user_id).unwrap_or(Uuid::nil());
+    let _ = sqlx::query(
+        "INSERT INTO ai_conversations (id, user_id, role, message, response) VALUES ($1, $2, $3, $4, $5)",
+    )
+    .bind(Uuid::new_v4())
+    .bind(user_id)
+    .bind(&req.user_role)
+    .bind(&message_text)
+    .bind(&response_text)
+    .execute(&state.db)
+    .await;
+
+    Ok(Json(ChatResponse {
+        response: response_text,
+    }))
+}
+
+// ── Jobs handler ───────────────────────────────────────────────
+
+async fn handle_jobs(
+    State(state): State<Arc<AppState>>,
+) -> Json<JobsResponse> {
+    Json(JobsResponse {
+        jobs: state.jobs.clone(),
+    })
+}
+
+// ── Health handler ─────────────────────────────────────────────
+
+async fn handle_health(
+    State(state): State<Arc<AppState>>,
+) -> Json<HealthResponse> {
+    Json(HealthResponse {
+        status: "ok".to_string(),
+        model: state.model.clone(),
+    })
+}
+
+// ── Main ───────────────────────────────────────────────────────
+
+#[tokio::main]
+async fn main() {
+    tracing_subscriber::fmt()
+        .with_env_filter(
+            tracing_subscriber::EnvFilter::try_from_default_env()
+                .unwrap_or_else(|_| "crm_ai=info,tower_http=info".into()),
+        )
+        .init();
+
+    dotenvy::dotenv().ok();
+
+    let database_url =
+        std::env::var("DATABASE_URL").expect("DATABASE_URL must be set");
+    let ollama_url =
+        std::env::var("OLLAMA_BASE_URL").unwrap_or_else(|_| "http://localhost:11434".to_string());
+    let model = std::env::var("AI_MODEL").unwrap_or_else(|_| "sam860/dolphin3-llama3.2:3b".to_string());
+    let host = std::env::var("AI_HOST").unwrap_or_else(|_| "0.0.0.0".to_string());
+    let port: u16 = std::env::var("AI_PORT")
+        .unwrap_or_else(|_| "3001".to_string())
+        .parse()
+        .expect("AI_PORT must be a number");
+
+    // Load jobs
+    let jobs_path = std::env::var("JOBS_PATH").unwrap_or_else(|_| "data/ai/jobs.jsonl".to_string());
+    let jobs_content = fs::read_to_string(&jobs_path).unwrap_or_default();
+    let jobs: Vec<Job> = jobs_content
+        .lines()
+        .filter(|l| !l.trim().is_empty())
+        .filter_map(|l| serde_json::from_str(l).ok())
+        .collect();
+
+    info!(
+        "Loaded {} job categories, model: {}, Ollama: {}",
+        jobs.len(),
+        model,
+        ollama_url
+    );
+
+    // Connect to database
+    let db = PgPoolOptions::new()
+        .max_connections(5)
+        .connect(&database_url)
+        .await
+        .expect("Failed to connect to database");
+
+    info!("Connected to PostgreSQL");
+
+    let state = Arc::new(AppState {
+        db,
+        ollama_url,
+        model,
+        jobs,
+    });
+
+    // CORS layer
+    let cors = CorsLayer::new()
+        .allow_origin(Any)
+        .allow_methods([Method::GET, Method::POST])
+        .allow_headers(Any);
+
+    let app = Router::new()
+        .route("/ai/chat", post(handle_chat))
+        .route("/ai/jobs", get(handle_jobs))
+        .route("/health", get(handle_health))
+        .layer(cors)
+        .with_state(state);
+
+    let addr = format!("{}:{}", host, port);
+    info!("CRM AI server listening on {}", addr);
+
+    let listener = tokio::net::TcpListener::bind(&addr)
+        .await
+        .expect("Failed to bind address");
+
+    // Start Facebook scraper in a separate thread
+    let rng = Arc::new(Mutex::new(StdRng::from_entropy()));
+    tokio::task::spawn_blocking(move || {
+        loop {
+            {
+                let _lock = rng.lock().unwrap();
+                run_facebook_scraper();
+            }
+            // Introduce random delay
+            let delay = {
+                let mut rng = rng.lock().unwrap();
+                rng.gen_range(1..5)
+            };
+            thread::sleep(Duration::from_secs(delay));
+        }
+    });
+
+    axum::serve(listener, app)
+        .await
+        .expect("Server failed");
+}