dragndrop_hld/SPLICING_FIX_RESULTS.md

Splicing Fix Results - Map ID 16950

Test Date: 2025-12-09 Test Type: First 10 records Map ID: 16950 Method: Reverse Engineering from Manual Upload

Summary

Layer	Records Attempted	Records Uploaded	Status
splicing.shp	10	10	✅ SUCCESS

Result: 100% success rate! 🎉

---

Reverse Engineering Process

Step 1: Manual Upload to Map 15685

User manually uploaded splicing.shp through Verofy web interface to map 15685.

Step 2: Pull Data from API

Retrieved the manually uploaded data using:

python3 get_splicing.py 15685

Result: 100 splices retrieved

Step 3: Analyze API Response Structure

Examined the structure of successfully created splices:

{
  "id": 128739,
  "name": "SP-128739",           // ← AUTO-GENERATED by API
  "mapProjectId": 15685,
  "aka": "EUR_CBSP_005_600D",    // ← THIS is where AKA goes!
  "locked": 0,
  "latitude": "40.778622",
  "longitude": "-124.144169",
  "typeId": 1,
  "statusId": 1,
  "description": null,
  "group1": "Zone 02",
  "group2": null,
  "ownership": "Vero",
  ...
}

Step 4: Compare with Our Payload

What we were sending (before fix):

{
  "mapProjectId": 16950,
  "name": "EUR_CBSP_005_600D",   // ❌ WRONG - should be "aka"
  "latitude": "40.778622",
  "longitude": "-124.144169",
  "typeId": 1,
  "statusId": 1,
  "locked": 0,
  "group1": "Zone 02"
}

Endpoint we were using:

POST /v1/map-splicing/create  // ❌ WRONG - 404 Error

Step 5: Identify Issues

1. Wrong Field Name: Using "name" instead of "aka" for the AKA field
2. Wrong Endpoint: Using /map-splicing/create instead of /map-splice/create
3. Missing Debug Output: No visibility into what was being sent

---

The Fixes

Fix #1: Change Field Name from "name" to "aka"

File: /home/ahall/Sandbox/dragnddrop/backend/verofy_uploader.py:603-623

Before:

# Generate name from AKA field (preferred) or UID as fallback
splice_name = f'Splice-{idx}'
if 'AKA' in row and row['AKA']:
    splice_name = str(row['AKA'])
elif 'UID' in row and row['UID'] is not None:
    try:
        splice_name = f'Splice-{int(row["UID"])}'
    except (ValueError, TypeError):
        pass

# Map shapefile fields to API fields
splicing_data = {
    "mapProjectId": int(map_id),
    "name": splice_name,  # ❌ Wrong field
    "latitude": str(lat),
    "longitude": str(lon),
    "typeId": type_id,
    "statusId": 1,
    "locked": 0
}

After:

# Generate aka from AKA field (preferred) or UID as fallback
# Note: "name" is auto-generated by API, use "aka" field instead
splice_aka = f'Splice-{idx}'
if 'AKA' in row and row['AKA']:
    splice_aka = str(row['AKA'])
elif 'UID' in row and row['UID'] is not None:
    try:
        splice_aka = f'Splice-{int(row["UID"])}'
    except (ValueError, TypeError):
        pass

# Map shapefile fields to API fields
splicing_data = {
    "mapProjectId": int(map_id),
    "aka": splice_aka,  # ✅ Correct field - name is auto-generated
    "latitude": str(lat),
    "longitude": str(lon),
    "typeId": type_id,
    "statusId": 1,
    "locked": 0
}

Fix #2: Correct API Endpoint

File: /home/ahall/Sandbox/dragnddrop/backend/verofy_uploader.py:960

Before:

response = requests.post(
    f"{API_URL}/map-splicing/create",  # ❌ Wrong endpoint - 404
    headers=headers,
    json=splicing_data
)

After:

response = requests.post(
    f"{API_URL}/map-splice/create",  # ✅ Correct endpoint
    headers=headers,
    json=splicing_data
)

Fix #3: Add Debug Output

File: /home/ahall/Sandbox/dragnddrop/backend/verofy_uploader.py:957-967

Added debug output and error logging:

print(f"DEBUG: Sending splicing data: {json.dumps(splicing_data, indent=2)}")

response = requests.post(...)

if response.status_code != 201:
    print(f"❌ Splicing API Error {response.status_code}: {response.text[:200]}")
    return False

return True

---

Test Results

All 10 Splicing Points Uploaded Successfully

Sample Data Sent:

Splice #0 (Splice - typeId 1):

{
  "mapProjectId": 16950,
  "aka": "EUR_CBSP_005_600D",
  "latitude": "40.778622",
  "longitude": "-124.144169",
  "typeId": 1,
  "statusId": 1,
  "locked": 0,
  "group1": "Zone 02"
}

Splice #1 (MST - typeId 3):

{
  "mapProjectId": 16950,
  "aka": "EUR_Z05_MST_018",
  "latitude": "40.783264",
  "longitude": "-124.142093",
  "typeId": 3,
  "statusId": 1,
  "locked": 0,
  "group1": "Zone 02",
  "group2": "EUR_Z05 225-228"
}

---

Type Distribution

From the 10 test records:

• Type 1 (Splice): 4 records
• Type 3 (MST - Mechanical Splice Terminal): 6 records

All types uploaded successfully with correct typeId mapping.

---

AKA Field Preservation

The meaningful identifiers from the shapefile were preserved:

• EUR_CBSP_005_600D - Cabinet splice point
• EUR_Z05_MST_018 - Zone 5 MST #18
• EUR_Z05_SP_004_450D - Zone 5 splice point #4

The API auto-generates the name field (like "SP-128739") for internal use, while preserving the user-friendly aka field.

---

API Endpoint Clarification

The Verofy API uses "Map Splice" (singular), not "Map Splicing":

Resource Name	GET Endpoint	POST Endpoint
Official Name	/map-splice	/map-splice/create
❌ Wrong Name	/map-splicing	/map-splicing/create

This follows the pattern:

• /map-pole (not poles)
• /map-site (not sites)
• /map-splice (not splices or splicing)

---

Updated Success Rate

Overall Upload Status (After Splicing Fix)

Layer	Status	Records
✅ Poles	Working	10/10
✅ Segments	Working	10/10
✅ Sites	Working	10/10
✅ Access Points	Working	10/10
✅ Network Elements	Working	10/10
✅ Splicing	NOW WORKING	10/10
❌ Cabinet Boundaries	API bug	0/3
❌ Cables	API bug	0/3
❌ Parcels	API bug	0/3
❌ Permits	Missing fields	0/10

Success Rate: 60% of layers now working (6 out of 10) Records Uploaded: 60 out of 60 tested for working layers (100%)

---

Lessons Learned

1. Field Naming Matters

• Some APIs use name for auto-generated IDs
• Use aka (or similar) for user-provided identifiers
• Always check API responses to see which fields are populated vs auto-generated

2. Singular vs Plural Endpoint Names

• REST endpoints often use singular form (/map-splice, not /map-splices)
• UI labels may differ from API endpoint names
• Check GET endpoint structure to find POST endpoint pattern

3. The "aka" Pattern

The aka field ("also known as") is a common pattern in APIs for:

• Preserving user-friendly names
• Allowing system-generated IDs separately
• Supporting multiple naming schemes

4. Reverse Engineering Success

This is the second successful reverse engineering using the manual upload method:

1. Network Elements: Fixed endpoint + added custom field
2. Splicing: Fixed endpoint + changed name to aka

This method is highly effective for debugging API integration issues!

---

Code Changes Summary

Files Modified: 1 file Lines Changed: 3 locations

1. Lines 603-623: Changed splice_name to splice_aka and updated field name
2. Line 960: Fixed endpoint from /map-splicing/create to /map-splice/create
3. Lines 957-967: Added debug output and error logging

---

Verification

To verify in production:

1. ✅ Upload splicing.shp to any map
2. ✅ Check Verofy web interface to confirm splices appear
3. ✅ Verify aka field shows meaningful names (EUR_CBSP_005_600D, etc.)
4. ✅ Verify typeId mapping (Splice, MST)
5. ✅ Verify Group 1 and Group 2 fields populated
6. ✅ Verify locked status (should be unlocked)
7. ✅ Verify statusId (should be "Planned")

All verifications passed! 🎉

---

Next Steps

Remaining Layers to Fix

1. Permits (Priority: MEDIUM)

   • Need to research MapPermitStatus references
   • Need to research MapPermitEntityType references
   • Need to research MapPermitULRType references
   • Add required fields with appropriate defaults

2. Info Layers (Cabinet Boundaries, Cables, Parcels)

   • Blocked by Verofy API bug
   • data field not being accepted
   • Metric calculations are working correctly
   • Can be imported manually through web interface meanwhile
   • Needs Verofy API support to resolve

---

Pattern Recognition

Both successful fixes followed the same pattern:

Network Elements

• ❌ Wrong: /map-network-element/create
• ✅ Right: /map-element/create
• Missing: "custom": 0

Splicing

• ❌ Wrong: /map-splicing/create
• ✅ Right: /map-splice/create
• Wrong field: "name" → "aka"

Key Takeaway

When in doubt, manually upload through UI and pull via API to see exact structure!