S3 AP access is authorized on the ONTAP side as the associated UNIX / Windows file system user, after IAM / S3 AP policy checks pass
The 5 Use Cases
UC1: Legal & Compliance — File Server Audit
Collects NTFS ACL information via ONTAP REST API, detects excessive permissions with Athena SQL, and generates natural-language compliance reports with Bedrock.
Business outcome:
- Reduce manual file-server audit effort
- Improve evidence collection for compliance reviews
Success metrics:
- Number of files scanned per run
- Time to generate audit report
- Percentage of files with missing or inconsistent metadata
Services: Athena, Glue Data Catalog, Bedrock | Verification: ✅ E2E success (67/67 Lambda executions)
UC2: Financial Services — Contract & Invoice Processing (IDP)
OCR processing of PDF/TIFF/JPEG documents with Textract, entity extraction with Comprehend, and structured summary generation with Bedrock.
Services: Textract, Comprehend, Bedrock | Verification: ✅ E2E success (Textract via cross-region invocation)
UC3: Manufacturing — IoT Sensor Log & Quality Inspection
CSV sensor logs converted to Parquet for Athena anomaly detection. Inspection images analyzed with Rekognition for defect detection with confidence-based manual review flagging.
Services: Athena, Glue Data Catalog, Rekognition | Verification: ✅ E2E success
UC4: Media — VFX Rendering Pipeline
Detects rendering assets, submits jobs to AWS Deadline Cloud, performs Rekognition quality checks, and writes approved output back to FSx ONTAP via S3 AP PutObject.
Services: Deadline Cloud, Rekognition | Verification: ✅ E2E success
UC5: Healthcare — DICOM Image Classification & Anonymization
Parses DICOM metadata for classification, detects burned-in PII with Rekognition DetectText, and removes PHI with Comprehend Medical.
Services: Rekognition, Comprehend Medical | Verification: ✅ E2E success (Comprehend Medical via cross-region)
Extension Patterns (Verified)
Bedrock Knowledge Bases — RAG
S3 AP as a data source for Bedrock Knowledge Bases. Verified with OpenSearch Serverless + Titan Embed Text v2 (81 documents indexed, Retrieve and RetrieveAndGenerate APIs confirmed).
Transfer Family SFTP — Partner File Exchange
SFTP server connected to S3 AP for external partner file exchange. Verified with SSH public key auth, upload/download operations.
EMR Serverless Spark — Large-Scale Processing
PySpark jobs reading/writing via S3 AP. Verified CSV → Parquet transformation with script and data I/O entirely through S3 AP.
Design Decisions
Shared Modules
All use cases share OntapClient (Secrets Manager auth, urllib3, TLS, retry), FsxHelper (AWS FSx API + CloudWatch metrics), S3ApHelper (pagination, suffix filter), and lambda_error_handler decorator.
Cost Optimization
High-cost always-on resources are opt-in via CloudFormation parameters:
| Resource |
Monthly Cost |
Default |
| Interface VPC Endpoints (4) |
~28ドル.80 |
Disabled |
| CloudWatch Alarms |
~0ドル.10/alarm |
Disabled |
| S3 Gateway VPC Endpoint |
Free |
Enabled |
Demo/PoC cost: ~1ドル–3ドル/month. Actual verification cost for all 8 patterns: under 2ドル.
Three-Layer Error Handling
-
Shared modules: Custom exceptions + urllib3/boto3 retry
-
Step Functions: Retry/Catch blocks with exponential backoff
-
Workflow: Map State individual failures don't affect other items
Cross-Region Invocation
Textract and Comprehend Medical are unavailable in some regions (e.g., ap-northeast-1). UC2 and UC5 use TextractRegion and ComprehendMedicalRegion CloudFormation parameters for cross-region API calls.
Note: Cross-region invocation transfers data to another region. Verify data residency and compliance requirements.
Issues Discovered During Verification
| # |
Issue |
Fix |
| 1 |
datetime JSON serialization |
Added default=str
|
| 2 |
Bedrock Messages API format |
Updated to Messages API |
| 3 |
Athena SQL quoting |
Added backtick quoting |
| 4 |
Lambda package name collision |
Added UC prefix to ZIP names |
| 5 |
S3 Gateway Endpoint duplication |
Added EnableS3GatewayEndpoint parameter |
| 6 |
VPC Lambda S3 AP timeout |
Added PrivateRouteTableIds parameter |
| 7 |
Textract region unavailability |
Added TextractRegion cross-region parameter |
| 8 |
ONTAP self-signed certificate |
Added VERIFY_SSL environment variable |
| 9 |
Single route table limitation |
Changed to CommaDelimitedList type |
| 10 |
Unnecessary VpcConfig |
Removed VpcConfig from S3 AP-only Lambda |
| 11 |
Comprehend Medical region |
Added ComprehendMedicalRegion parameter |
| 12 |
UC4 QualityCheck KeyError |
Safe key access pattern |
| 13 |
pyarrow Lambda layer size |
Replaced with stdlib csv module |
When to Use / When Not to Use
Use this when:
- You want to serverlessly process existing NAS data on FSx for ONTAP without moving it
- You need file listing and preprocessing from Lambda without NFS/SMB mounts
- You want to learn the separation of responsibilities between S3 AP and ONTAP REST API
- You want to quickly validate industry-specific AI/ML patterns as a PoC
Don't use this when:
- Real-time file change event processing is required (S3 Event Notification not supported)
- Full S3 bucket compatibility (Presigned URLs, etc.) is needed
- You already have EC2/ECS batch infrastructure with NFS mount operations
- File data already exists in standard S3 buckets
Production Readiness Considerations
This repository includes production-oriented design decisions, but actual production environments should additionally consider:
- Organizational IAM / SCP / Permission Boundary alignment
- S3 AP policy and ONTAP-side user permission review
- Audit and execution logs (CloudTrail / CloudWatch Logs)
- CloudWatch Alarms / SNS / Incident Management integration
- Industry-specific compliance (data classification, PII, PHI)
- Data residency for cross-region invocations
Getting Started
git clone https://github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns.git
cd FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns
pip install -r requirements.txt
pip install -r requirements-dev.txt
pytest shared/tests/ -v
# Package and deploy (example: UC1)
export AWS_DEFAULT_REGION=us-east-1
./scripts/deploy_uc.sh legal-compliance package
# Then deploy via CloudFormation — see README for full parameter list
The repository includes 8-language READMEs (ja, en, ko, zh-CN, zh-TW, fr, de, es), deployment guides, operations guides, troubleshooting guides, cost analysis, and region compatibility matrix.
Repository: github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns
Yoshiki Fujiwara
📢 Update (2026年05月23日)
This article is part of the FSx for ONTAP S3 Access Points series.
The latest addition — Phase 13: From Serverless Patterns to Field-Ready Reference Architecture — is now available:
👉 Read Phase 13
Phase 13 adds FlexCache/FlexClone serverless automation, split-path S3AP monitoring, SLO runbooks, Partner/SI delivery checklist, and a complete field-ready baseline for informed evaluation.