dlackner · dlackner · Mar 15, 2026 · Mar 15, 2026 · Mar 15, 2026
diff --git a/admin/monthly_rental_process_README.md b/admin/monthly_rental_process_README.md
@@ -10,17 +10,26 @@ This directory contains the automated monthly rental data processing system for
 
 ## Monthly Process
 
-### 1. Receive CSV File - put file in public folder and give Claude the filename.  It will take it from there
-Each month, you'll receive a Zillow MSA rental data CSV file with format:
-- Columns: RegionID, SizeRank, RegionName, RegionType, StateName, 2015-01-31, 2015-02-28, ...
-- Latest column contains the current month's data (e.g., 2025-08-31)
+### 1. Prepare CSV File
+Each month, you'll receive a Zillow MSA rental data CSV file with the following format:
+```
+RegionID,SizeRank,City/State,Lat,Long,Monthly Average,Radius,YOY
+394913,1,"New York, NY",40.7128,-74.006,3232,25.0,4.28%
+...
+```
 
 ### 2. Run Processing Script
 ```bash
 cd /home/dan/Work/mfos/admin
-python3 monthly_rental_data_processor.py /path/to/new_csv_file.csv
+python3 monthly_rental_data_processor.py path/to/csv_file.csv
 ```
 
+The script will:
+- Process the CSV data
+- Add market tier assignments
+- Generate SQL for database import
+- Place output CSV in `/public/Monthly Rental Rates.csv` (if enabled)
+
 ### 3. Review Generated Files
 The script automatically generates:
 - `rental_data_upsert_YYYY_MM_DD.sql` - SQL file for database import
@@ -29,26 +38,26 @@ The script automatically generates:
 ### 4. Import to Database
 Execute the SQL file in your Supabase database to update the `market_rental_data` table.
 
+**Note:** The map component now reads directly from the database via `/api/rental-data`, so once the SQL is executed, the map displays current data automatically.
+
 ## Features
 
-### ✅ Automated Processing
-- **Auto-detects target month** from CSV columns
-- **6-month lookback** for missing current data
-- **YOY calculations** based on actual dates used (12 months back)
-- **Market tier assignment** (1=Primary, 2=Secondary, 3=Tertiary)
-- **Coordinate lookup** for 500+ cities
+### ✅ Data Processing
+- **Parses rental CSV data** with RegionID, City/State, rental values
+- **Market tier assignment** (1=Primary, 51-100=Secondary, 101+=Tertiary)
+- **Coordinate lookup** for 720+ MSAs
+- **YOY parsing** from CSV year_over_year_growth column
 
 ### ✅ Data Quality
 - **Preserves existing coordinates** via COALESCE in UPSERT
-- **Handles missing data** gracefully with fallback logic
-- **Validates data types** and filters MSA-only records
-- **Reports missing coordinates** for manual lookup
+- **Validates coordinates** - reports missing coordinates
+- **SQL conflict handling** - updates or inserts based on region_id
+- **Reports processing statistics** - tier breakdown, coordinate coverage
 
 ### ✅ Business Rules
 - **Market Tiers**: 1-50=Tier 1, 51-100=Tier 2, 101+=Tier 3
-- **Default Radius**: T1=50mi, T2=35mi, T3=10mi
-- **YOY Calculation**: Dynamic based on actual data dates
-- **Missing Data**: 6-month lookback, preserves existing data
+- **Default Radius**: Varies by market (typically 7.5-25 miles)
+- **YOY Storage**: Stored as text (e.g., '4.28%') and numeric value
 
 ## Database Schema
 
@@ -75,68 +84,82 @@ market_rental_data (
 ### Summary Report
 ```
 === MONTHLY RENTAL DATA PROCESSING SUMMARY ===
-Processing Date: 2025-10-01 15:49:43
-Target Month: 2025-08-31
-Total MSAs processed: 576
+Processing Date: 2026-03-15 11:00:44
+Target Month: 2026-01-31
+Total MSAs processed: 720
 
 📊 MARKET TIER BREAKDOWN:
-  Tier 1 (Primary): 49 markets
+  Tier 1 (Primary): 50 markets
   Tier 2 (Secondary): 50 markets
-  Tier 3 (Tertiary): 477 markets
+  Tier 3 (Tertiary): 620 markets
 
 📈 YOY GROWTH STATISTICS:
-  MSAs with YOY data: 421
-  MSAs missing YOY: 155
-  Average YOY growth: 3.36%
-
-🚀 TOP 10 FASTEST GROWING MARKETS:
-   1. Abilene, TX: 19.92%
-   2. Mankato, MN: 14.00%
-   [...]
+  MSAs with YOY data: 720
+  MSAs missing YOY: 0
+  Average YOY growth: 2.14%
 
 🗺️ COORDINATE COVERAGE:
-  Cities with coordinates: 538 (93.4%)
-  Cities missing coordinates: 38
+  Cities with coordinates: 720 (100%)
+  Cities missing coordinates: 0
 ```
 
 ### SQL File Structure
 ```sql
 -- Auto-generated UPSERT for market rental data
+-- Processed on 2026-03-15 11:00:44
+-- Total MSAs: 720
+-- Cities with coordinates: 720
+
 INSERT INTO public.market_rental_data
-  (region_id, size_rank, city_state, latitude, longitude, ...)
+  (region_id, size_rank, city_state, latitude, longitude, monthly_rental_average, radius, year_over_year_growth, yoy_growth_numeric, market_tier, updated_at)
 VALUES
-  (394913, 1, 'New York, NY', 40.7128, -74.006, 3555, 50.0, '5.03%', 5.03, 1, now()),
+  (394913, 1, 'New York, NY', 40.7128, -74.006, 3232, 25.0, '4.28%', 4.28, 1, now()),
+  (753899, 2, 'Los Angeles, CA', 34.0522, -118.2437, 2885, 25.0, '1.64%', 1.64, 1, now()),
   [...]
 ON CONFLICT (region_id) DO UPDATE
-SET [updates with COALESCE protection]
+SET
+  size_rank = EXCLUDED.size_rank,
+  city_state = EXCLUDED.city_state,
+  monthly_rental_average = EXCLUDED.monthly_rental_average,
+  year_over_year_growth = EXCLUDED.year_over_year_growth,
+  yoy_growth_numeric = EXCLUDED.yoy_growth_numeric,
+  market_tier = EXCLUDED.market_tier,
+  updated_at = now();
 ```
 
+## Workflow
+
+### Complete Monthly Update Process
+1. **Receive CSV** - Get monthly rental data from Zillow
+2. **Process CSV** - Run `python3 monthly_rental_data_processor.py filename.csv`
+3. **Review Output** - Check summary report for coordinate coverage and statistics
+4. **Execute SQL** - Run the generated `rental_data_upsert_YYYY_MM_DD.sql` in Supabase
+5. **Verify in App** - Map automatically shows updated data from `/api/rental-data`
+
+### How the App Uses This Data
+- **Map Component** (`/discover` page) fetches from `/api/rental-data`
+- **API Endpoint** queries `market_rental_data` table directly
+- **Data displays** as rental circles with color-coded rent levels
+- **Popup shows** city name, average rent, market rank, and YOY growth
+
 ## Troubleshooting
 
 ### Missing Coordinates
 If new cities appear without coordinates:
-1. Note the region IDs from the summary report
+1. Note region IDs from the summary report
 2. Look up coordinates manually
-3. Add to the `get_coordinate_lookup()` function
+3. Add to `get_coordinate_lookup()` function in the Python script
 4. Re-run the processing script
 
-### Data Issues
-- Check CSV format matches expected structure
-- Verify RegionType = 'msa' for proper filtering
-- Ensure date columns follow YYYY-MM-DD format
-
-## Maintenance
-
-### Monthly Tasks
-1. Run processing script with new CSV
-2. Review summary for data quality
-3. Execute SQL in Supabase
-4. Verify import success
+### Data Quality Issues
+- Verify CSV column format: `RegionID,SizeRank,City/State,Lat,Long,Monthly Average,Radius,YOY`
+- Check that rental values are numeric (not text)
+- Ensure YOY column contains percentages (e.g., '4.28%') or blank
 
-### Periodic Tasks
-- Update coordinate lookup for new cities
-- Monitor YOY calculation accuracy
-- Review market tier assignments
+### Map Not Updating
+- Verify SQL was executed in Supabase (`market_rental_data` table)
+- Check `/api/rental-data` endpoint in browser Network tab
+- Hard refresh browser (Ctrl+Shift+R) to clear caches
 
 ## Support
 

diff --git a/app/api/rental-data/route.ts b/app/api/rental-data/route.ts
@@ -0,0 +1,51 @@
+/*
+ * CHARLIE2 V2 - Rental Data API
+ * Serves market rental data from database for map overlays
+ * Replaces CSV-based approach with real-time database queries
+ */
+
+import { NextResponse } from 'next/server';
+import { createClient } from '@supabase/supabase-js';
+
+const supabase = createClient(
+  process.env.NEXT_PUBLIC_SUPABASE_URL!,
+  process.env.SUPABASE_SERVICE_ROLE_KEY!
+);
+
+export async function GET() {
+  try {
+    // Fetch all rental data from database
+    const { data: rentalData, error } = await supabase
+      .from('market_rental_data')
+      .select('region_id, size_rank, city_state, latitude, longitude, monthly_rental_average, radius, year_over_year_growth, yoy_growth_numeric')
+      .order('size_rank', { ascending: true });
+
+    if (error) {
+      console.error('Error fetching rental data:', error);
+      return NextResponse.json({ error: 'Failed to fetch rental data' }, { status: 500 });
+    }
+
+    if (!rentalData) {
+      return NextResponse.json([], { status: 200 });
+    }
+
+    // Transform database format to CSV format for compatibility with RentDataProcessor
+    const csvContent = [
+      'RegionID,SizeRank,City/State,Lat,Long,Monthly Average,Radius,YOY %',
+      ...rentalData.map(row =>
+        `${row.region_id},${row.size_rank},"${row.city_state}",${row.latitude},${row.longitude},${row.monthly_rental_average},${row.radius},"${row.year_over_year_growth || ''}"`
+      )
+    ].join('\n');
+
+    return new NextResponse(csvContent, {
+      headers: {
+        'Content-Type': 'text/csv; charset=utf-8',
+        'Cache-Control': 'public, max-age=3600', // Cache for 1 hour
+        'ETag': `"rental-data-${Date.now()}"` // Simple ETag for cache busting
+      }
+    });
+  } catch (error) {
+    console.error('Error in rental data API:', error);
+    return NextResponse.json({ error: 'Internal server error' }, { status: 500 });
+  }
+}
diff --git a/app/page.tsx b/app/page.tsx
@@ -8,7 +8,7 @@
 
 import { useState } from 'react';
 import { useAuth } from '@/contexts/AuthContext';
-import { ChevronLeft, ChevronRight, TrendingUp, FileText, Mail, DollarSign, Building, Users, Target, Zap, Globe, Brain, BarChart3, MessageSquare, Calendar, CheckCircle, X, Crown, Play } from 'lucide-react';
+import { ChevronLeft, ChevronRight, TrendingUp, FileText, Mail, DollarSign, Building, Users, Target, Zap, Globe, Brain, BarChart3, MessageSquare, Calendar, CheckCircle, X, Crown, Play, AlertTriangle } from 'lucide-react';
 import Image from 'next/image';
 import { Dialog } from '@headlessui/react';
 import TypewriterChatDemo from '@/components/ui/TypewriterChatDemo';
@@ -357,17 +357,44 @@ export default function Home() {
                   </button>
                 </form>
               ) : (
-                <div className="text-center p-4 bg-green-50 border border-green-200 rounded-xl">
-                  <CheckCircle className="h-8 w-8 text-green-600 mx-auto mb-2" />
-                  <p className="text-green-600 font-medium mb-4">
-                    Check your email for the confirmation link!
-                  </p>
-                  <p className="text-sm text-gray-600">
-                    We've sent a login link to <strong>{signupEmail}</strong>. Click the link in your email to complete your registration and start your 7-day free trial.
-                  </p>
-                  <p className="text-xs text-gray-500 mt-3">
-                    Didn't receive the email? Check your spam folder.
-                  </p>
+                <div className="bg-white rounded-lg border border-gray-200 p-6">
+                  {/* Success Section */}
+                  <div className="flex items-start space-x-4 mb-6">
+                    <div className="flex-shrink-0">
+                      <div className="w-10 h-10 bg-blue-100 rounded-lg flex items-center justify-center">
+                        <CheckCircle className="w-5 h-5 text-blue-600" />
+                      </div>
+                    </div>
+                    <div className="flex-1">
+                      <h3 className="text-lg font-bold text-gray-900">Check Your Email</h3>
+                      <p className="text-sm text-gray-600">MultifamilyOS.ai</p>
+                      <p className="text-sm text-gray-600 mt-2">
+                        A confirmation link has been sent to <strong>{signupEmail}</strong>
+                      </p>
+                    </div>
+                  </div>
+
+                  {/* Warning Section */}
+                  <div className="border-t border-gray-200 pt-6">
+                    <div className="flex items-start space-x-4">
+                      <div className="flex-shrink-0">
+                        <div className="w-10 h-10 bg-amber-100 rounded-lg flex items-center justify-center">
+                          <AlertTriangle className="w-5 h-5 text-amber-600" />
+                        </div>
+                      </div>
+                      <div className="flex-1">
+                        <h4 className="font-bold text-gray-900 mb-2">Check Your Spam Folder</h4>
+                        <p className="text-sm text-gray-600 mb-3">
+                          Our confirmation email often gets filtered. If you don't see it in your inbox, check:
+                        </p>
+                        <ul className="text-sm text-gray-600 space-y-1 text-left">
+                          <li>• <strong>Gmail:</strong> Check "Promotions" or "Spam" tabs</li>
+                          <li>• <strong>Outlook:</strong> Check your "Junk" folder</li>
+                          <li>• <strong>Other providers:</strong> Search for "MultifamilyOS"</li>
+                        </ul>
+                      </div>
+                    </div>
+                  </div>
                 </div>
               )}
             </div>
@@ -707,16 +734,44 @@ export default function Home() {
                 </p>
               </div>
             ) : (
-              <div className="text-center p-4 bg-green-50 border border-green-200 rounded-xl">
-                <CheckCircle className="h-8 w-8 text-green-600 mx-auto mb-2" />
-                <h3 className="font-medium text-green-800 mb-1">Check Your Email</h3>
-                <p className="text-green-600 text-sm">
-                  A confirmation link has been sent to <strong>{email}</strong>. 
-                  Click the link to complete your registration and start your 7-day free trial.
-                </p>
-                <p className="text-xs text-gray-500 mt-3">
-                  Didn't receive the email? Check your spam folder.
-                </p>
+              <div className="bg-white rounded-lg border border-gray-200 p-6">
+                {/* Success Section */}
+                <div className="flex items-start space-x-4 mb-6">
+                  <div className="flex-shrink-0">
+                    <div className="w-10 h-10 bg-blue-100 rounded-lg flex items-center justify-center">
+                      <CheckCircle className="w-5 h-5 text-blue-600" />
+                    </div>
+                  </div>
+                  <div className="flex-1">
+                    <h3 className="text-lg font-bold text-gray-900">Check Your Email</h3>
+                    <p className="text-sm text-gray-600">MultifamilyOS.ai</p>
+                    <p className="text-sm text-gray-600 mt-2">
+                      A confirmation link has been sent to <strong>{email}</strong>
+                    </p>
+                  </div>
+                </div>
+
+                {/* Warning Section */}
+                <div className="border-t border-gray-200 pt-6">
+                  <div className="flex items-start space-x-4">
+                    <div className="flex-shrink-0">
+                      <div className="w-10 h-10 bg-amber-100 rounded-lg flex items-center justify-center">
+                        <AlertTriangle className="w-5 h-5 text-amber-600" />
+                      </div>
+                    </div>
+                    <div className="flex-1">
+                      <h4 className="font-bold text-gray-900 mb-2">Check Your Spam Folder</h4>
+                      <p className="text-sm text-gray-600 mb-3">
+                        Our confirmation email often gets filtered. If you don't see it in your inbox, check:
+                      </p>
+                      <ul className="text-sm text-gray-600 space-y-1 text-left">
+                        <li>• <strong>Gmail:</strong> Check "Promotions" or "Spam" tabs</li>
+                        <li>• <strong>Outlook:</strong> Check your "Junk" folder</li>
+                        <li>• <strong>Other providers:</strong> Search for "MultifamilyOS"</li>
+                      </ul>
+                    </div>
+                  </div>
+                </div>
               </div>
             )}
           </div>

diff --git a/components/shared/PropertyMapWithRents.tsx b/components/shared/PropertyMapWithRents.tsx
@@ -30,24 +30,24 @@ export default function PropertyMapWithRents(props: PropertyMapWithRentsProps) {
   const [rentData, setRentData] = useState<ProcessedRentData[]>([]);
   const [isLoadingRentData, setIsLoadingRentData] = useState(true);
 
-  // Load rental data from CSV file on component mount (same as legacy version)
+  // Load rental data from API on component mount
   useEffect(() => {
     const loadRentData = async () => {
       try {
         setIsLoadingRentData(true);
-        
-        // Fetch rental data from CSV file (same as legacy my-properties)
-        const response = await fetch('/Monthly Rental Rates.csv?v=3');
+
+        // Fetch rental data from database API
+        const response = await fetch('/api/rental-data');
         if (!response.ok) {
-          throw new Error(`Failed to fetch CSV: ${response.status}`);
+          throw new Error(`Failed to fetch rental data: ${response.status}`);
         }
-        
+
         const csvText = await response.text();
-        
-        // Use same RentDataProcessor as legacy version
+
+        // Use same RentDataProcessor for CSV format compatibility
         const processor = new RentDataProcessor(csvText);
         const processedData = processor.processRentData();
-        
+
         setRentData(processedData);
       } catch (error) {
         console.error('Error loading rent data:', error);