Troubleshoot Spanner Vertex AI integration
Stay organized with collections
Save and categorize content based on your preferences.
MODEL DDL statements and ML functions in Spanner invoke Vertex AI endpoints and can fail due to various reasons:
| Error Code | Error Message | Possible cause | Possible solution |
|---|---|---|---|
| CANCELLED | Call to Vertex AI endpoint {ENDPOINT} was cancelled due to query cancellation. | Query was cancelled by the client application. | Investigate why your client cancelled the query. |
| DEADLINE_EXCEEDED | Vertex AI endpoint {ENDPOINT} exceeded the call deadline. | Query deadline is too short. | Increase query deadline on the client. |
| - | - | Endpoint was too busy. | See Vertex AI monitoring and deploy more nodes. |
| FAILED_PRECONDITION | Vertex AI endpoint {ENDPOINT} returned failed precondition error. | Endpoint has no models deployed. | Deploy models to the endpoint. | INTERNAL | Unknown error when accessing Vertex AI endpoint {ENDPOINT}. | Unexpected internal error. | Use failover endpoints or open a support ticket. |
| INVALID_ARGUMENT | Invalid request to Vertex AI endpoint {ENDPOINT}. Make sure that Vertex AI endpoint and Spanner model schema match. | Spanner model schema and Vertex AI endpoint schema do not match | Update Spanner model's schema. |
| NOT_FOUND | Vertex AI endpoint {ENDPOINT} not found. | Endpoint was deleted. | Update Spanner model's schema. |
| PERMISSION_DENIED | Access to Vertex AI endpoint {ENDPOINT} was denied. | Spanner service agent does not have permissions to access the endpoint | Grant service agent role permissions |
| - | - | VPC SC error | See Vertex AI error message and follow VPC SC troubleshooting | RESOURCE_EXHAUSTED | Vertex AI endpoint {ENDPOINT} quota has been exceeded. | Too many requests to Vertex AI. | Increase online prediction quota | UNAVAILABLE | Could not create service agent for project {PROJECT}. | Service Usage API issue. | Create service agent manually | - | Vertex AI endpoint {ENDPOINT} is unavailable. | Too many requests to Vertex AI. | Deploy more nodes. | - | - | Vertex AI has a regional issue | Use failover endpoints |