# Phase 4 Summary — The Complete Marketing System

Built on Phases 1-3 without redesigning anything: one additive
migration (`0015`), the same MVC conventions, the same AJAX-envelope
pattern every page already used. This phase is unusual in one respect:
several of its tables (`email_templates`, `campaigns`,
`campaign_recipients`, `automation_workflows`, `automation_steps`,
`automation_runs`) have existed, empty, since Phase 1 — Phase 4 is
mostly new *application code* finally putting them to work, plus a
handful of genuinely new tables for things Phase 1 didn't anticipate
(the send queue, the global unsubscribe list, saved audience segments,
SMTP settings).

Everything below was run against a live MySQL 8 database and a real
PHP server during development — not just written and assumed correct.

## What was built

### Modules
| Module | Where |
|---|---|
| SMTP Settings | `SmtpSettingsModel` (AES-256-CBC encrypted password) + `SmtpSettingsController` — admin-configurable, `MailService` prefers it over `.env` when present |
| PHPMailer | `MailService` (Phase 1), now resolving its transport config from the DB first |
| Email Templates | `EmailTemplateModel` + `EmailTemplateController` — full CRUD, merge-field rendering (`{{first_name}}` etc.) |
| Email Queue | `email_queue` table + `EmailQueueModel`/`EmailQueueService` — decouples "queued" from "sent," `bin/process_email_queue.php` worker |
| Bulk Email | `CampaignService::sendNow()`/`schedule()` — resolves an audience, creates real `campaign_recipients` rows, queues every send |
| Personal Email | `PersonalEmailController` — immediate synchronous send to one lead/customer, logged to `email_messages`, surfaced in that record's new "Emails" tab |
| Campaign Builder | `marketing/campaigns.php` — chip-based audience picker (All Leads / Homecare / Cleaner / Customers / Saved Segment / Filtered Audience), live recipient-count preview, template starter, recurrence toggle |
| Campaign Scheduler | `CampaignController::schedule()` — same dispatch path as Send Now, just with a future `scheduled_for` on the queued rows |
| Audience Segments | `AudienceSegmentService` (resolution) + `AudienceSegmentModel`/`AudienceSegmentController` (saved definitions) |
| Marketing Automation | `AutomationEngine` — a real execution engine for Phase 1's `automation_workflows`/`automation_steps`/`automation_runs` schema |
| Newsletter | Recurring campaigns (`campaigns.is_recurring_parent` + `recurrence_rule`) — same real-child-row pattern as Phase 3's recurring appointments |
| Notifications | Existing `NotificationService` (Phase 2) — campaign completion could hook in the same way `ActivityLogger` already does; see Phase 5 readiness |

### Features
| Feature | Where |
|---|---|
| Send Individual Email | `PersonalEmailController::send()` |
| Send Bulk Email | `CampaignService::sendNow()`/`schedule()` |
| Send to: All Leads / Homecare / Cleaner / Both / Customers / Filtered Audience | `AudienceSegmentService::resolve()` — one method, a small JSON filter DSL, verified live for every one of these six cases |
| Track: Delivery | `campaign_recipients.sent_at` (immutable fact, not overwritten by later status changes — see bug #2 below) |
| Track: Open Rate | 1x1 tracking pixel, `TrackingController::pixel()` calling `TrackingService::recordOpen()` |
| Track: Click Rate | Click-through redirect, `TrackingController::click()` calling `TrackingService::recordClick()` |
| Track: Bounce Rate | `TrackingService::recordBounce()` — admin/webhook-ready, not wired to a real ESP webhook yet (see Phase 5 readiness) |
| Track: Replies | `TrackingService::recordReply()` — same webhook-ready status as bounces |
| Track: Unsubscribe | Public `/unsubscribe/{token}` page, `TrackingService::recordUnsubscribe()` — writes to the GLOBAL `email_unsubscribes` list, not just the one campaign |

### Campaigns
Draft, Scheduled, Sending, Completed (`status='sent'` — see "Design
decisions" below), Cancelled — all real status transitions, verified
live: draft leads to sendNow() leads to sending leads to (queue
processed) leads to sent, automatically, with no separate "mark as
sent" step. Recurring campaigns are Newsletter.

### Automation Rules (the 5 named in the brief)
All 5 are seeded, real, active workflows — not mocked:
1. **Welcome Email** — `lead_created` trigger, one `send_email` step.
2. **Follow-up Email** — `lead_created` trigger, `wait` (72h) then
   `send_email`. This is the one that proved the engine's hardest
   case: fire, immediate first step, pause at `wait`, (time passes),
   worker resumes, second step executes, run completes. Verified
   live end-to-end, including forcing `next_step_at` into the past to
   simulate the wait elapsing.
3. **Appointment Reminder** — intentionally NOT a generic-engine
   workflow. It's executed by `AppointmentService::sendDueReminders()`
   (Phase 3), which already reads `reminder_minutes_before` directly;
   a workflow row exists purely so it's visible in the Automation admin
   list, with a `trigger_config` note explaining why. Duplicating it as
   generic steps would just be two implementations of the same thing.
4. **Birthday Email** — `customer_birthday` trigger, checked daily by
   `AutomationEngine::checkDateBasedTriggers()` (no natural "event" to
   fire from), deduped so it only fires once per year per customer.
5. **Re-engagement Campaign** — `customer_inactive_90d` trigger, same
   daily-check pattern, fires for any active customer with no
   appointment in the last 90 days.

## What was actually tested (not just written)

- **SMTP settings encryption**: saved a password, confirmed it decrypts
  back to the exact original string, confirmed the encrypted value at
  rest is NOT the plaintext, and confirmed that re-saving settings
  without a new password correctly preserves the previously encrypted
  one rather than wiping it.
- **Audience segmentation**: created 3 Homecare and 2 Janitorial test
  leads, unsubscribed one, and confirmed all six send-to targets
  resolved to the exact right counts — including that the unsubscribed
  lead was correctly excluded from "All Leads" and "Homecare Leads"
  alike.
- **Full campaign lifecycle**: created a draft, called `sendNow()`,
  confirmed `campaign_recipients` rows and `email_queue` rows were
  created correctly, ran the actual queue worker (against a
  purpose-built fake PHPMailer transport, since this sandbox has no
  real SMTP relay — see "Testing without real SMTP" below), confirmed
  the campaign auto-transitioned from `sending` to `sent` once every
  queued email cleared, then simulated an open/click/unsubscribe and
  confirmed the campaign's rollup stats and the tracking rates were
  exactly right.
- **Automation engine**: built a 3-step test workflow
  (send_email, wait, send_email), fired it, confirmed the first
  email queued immediately and the run paused with the correct
  `next_step_at`, forced time forward, confirmed the worker resumed it
  and queued the second email, and confirmed re-firing the same
  trigger for the same entity within 24h was correctly deduplicated.
- **Full HTTP flow**: real login, create a lead (confirmed this
  actually fired both the Welcome Email and Follow-up Email workflows
  for real, observable in `email_queue` and `automation_runs`),
  preview an audience count, create and send a campaign, confirm the
  merge-personalized subject line landed in the queue correctly,
  publicly (no auth) hit the unsubscribe page and confirm it, publicly
  hit the open-tracking pixel and confirm the correct GIF content-type
  and DB update, confirm campaign stats rendered on the Dashboard.

### Testing without real SMTP
This sandbox has no outbound network access to a real mail relay and
PHPMailer isn't installed (no packagist access to `composer require`
it). Rather than skip testing the queue/send logic, a minimal
stand-in `PHPMailer\PHPMailer\PHPMailer` class was defined for test
runs only (not shipped) so `MailService`'s real code path — config
resolution, property assignment, the actual `send()` call — ran
against a fake transport instead of being bypassed. `MailService`
itself is unmodified from this; a real deploy just needs
`composer require phpmailer/phpmailer` and either `.env` or the SMTP
Settings page filled in.

### Bugs caught by testing, not assumed away
1. **Campaign subject lines didn't get merge-field rendering.** Only
   the body went through `EmailTemplateModel::renderMergeFields()`; a
   campaign with `{{first_name}}` in the subject would have literally
   sent that placeholder text to every recipient. Caught by inspecting
   the actual queued row's `subject` column after a real send, not by
   reading the code — fixed and re-verified live (subject correctly
   read "Hello Marketing" instead of "Hello {{first_name}}").
2. **`delivery_rate` was computed from current `status`, not the
   immutable `sent_at` fact.** A recipient who was successfully
   delivered and later unsubscribed had their status overwritten from
   `sent` to `unsubscribed`, making them silently vanish from the
   delivery-rate numerator even though delivery genuinely happened.
   Fixed to key off `sent_at IS NOT NULL`, which nothing ever
   overwrites — verified live: the rate went from an incorrect 66.7%
   to the correct 100%.

## Design decisions worth flagging explicitly

- **"Completed" is `campaigns.status = 'sent'`**, not a new enum value.
  The brief lists Draft/Scheduled/Recurring/Completed/Cancelled;
  Phase 1's schema already had `draft/scheduled/sending/sent/cancelled`.
  Rather than add a redundant `completed` value, the UI labels `sent`
  as "Completed" — same underlying fact, no schema churn. "Recurring"
  isn't a status either; it's `is_recurring_parent = 1` on a campaign
  whose own status still progresses normally.
- **One opaque token per recipient**, not three. `unsubscribe_token`
  does triple duty as the open-pixel token and click-redirect token
  too — an unguessable 64-char random string is exactly as secure for
  all three purposes, and minting three separate tokens per recipient
  would just be more surface area for the same protection.
- **Bounce and reply tracking are webhook-ready, not webhook-wired.**
  `TrackingService::recordBounce()`/`recordReply()` exist, are called
  correctly by an admin action today, and are exactly the methods a
  real ESP webhook (SES/Mailgun/SendGrid bounce notifications, or
  IMAP-based inbound reply parsing) would call once one is configured
  — but standing up that integration is infrastructure work outside
  what "build the marketing system" means for this phase.

## Phase 5 readiness (AI)

Nothing here needs revisiting to build AI modules:

1. **`conversations`/`conversation_messages`/`chatbot_settings`**
   (Phase 1 schema) are completely untouched and ready for an AI
   chatbot/voice-agent module to follow the exact same
   Controller-Service-Model pattern every module in Phases 1-4 has
   used.
2. **`AutomationEngine`'s `webhook` step type** is currently a logged
   stub (`executeWebhookStub()`) — this is the natural extension point
   for an AI module to plug a step into a workflow (e.g. "have the AI
   assistant draft a personalized follow-up" as a step between `wait`
   and `send_email`) without changing the engine's core advance/pause
   logic at all.
3. **`AudienceSegmentService`'s filter DSL** is a small, deliberately
   simple JSON structure specifically so an AI assistant could
   generate one from a natural-language prompt ("email everyone who's
   been quoted but hasn't booked in 2 weeks") and hand it straight to
   `resolve()` — no new resolution logic needed, just a translator in
   front of it.
4. **Email Queue and Campaign infrastructure** are exactly what an
   AI-drafted campaign or AI-triggered automation would use to actually
   send — nothing about "the AI decided what to say" changes how it
   gets delivered, tracked, or reported on.