Simple multi-site or hybrid cloud workflow for S3 analytics with ONTAP S3
In this video I use the native S3 bucket created in the earlier video at
https://rumble.com/v32fmic-brief-walk-through-over-ontap-native-and-multi-protocol-s3-services.html
The idea is to show how one could organize multi-location ingress and processing:
- Site A generates or ingresses data via S3 or NFS (in the case of multi-protocol ONTAP S3 buckets)
- We use SnapMirror S3 to replicate to another ONTAP system such as AWS FSxN in the cloud. For multi-protocol buckets we could use SnapMirror (not SnapMirror S3!), CloudSync, rclone or some other utility
- Site B can read data using NFS (multi-protocol) or S3 - whichever works better. It is recommended to use the same config (either multi-protocol, or "pure" S3) on both sides to avoid incompatibility issues.
The video shows an example with native S3 buckets on both sides.
On-the-fly conversion from Parquet file to Panda dataframes is meant to show sometimes data doesn't even need to be copied off S3 to local disk to be converted, which is convenient as the clients don't even need to mount NFS.
The video is a bit short for so many steps, but you can check ONTAP S3-related and analytics-related solutions documentation for more comprehensive descriptions of such workflows.
-
3:04
scaleoutSean's demos and how-to's
11 months agoBrief walk-through over ONTAP "native" and multi-protocol S3 services
7 -
0:50
Affiliate Marketing Reviews And Videos
1 year agoMake Ultimate Limitless Cloud Storage Portal | SelfCloud Pro Review
15 -
4:36
ryantvackner
1 year agoThe Biggest Challenge in Data Analytics
2 -
1:35:00
CodeWithYu
8 months agoReddit Data Pipeline | AWS End to End Data Engineering
16 -
1:10
bharani12
1 year agoTop 9 Data Analytics Tools for 2022
7 -
4:40
ideadudes
3 years agoAzure Monitor - Metrics Uses and Configuration
73 -
7:44
EvantheTerrrible
2 years ago $0.01 earnedBuilding an Enterprise Reporting Architecture with Azure, KNIME, and Power BI
206 -
5:22
WorldWiseNow
7 months agoCloud-First, Data-Driven Transformation
2 -
58:05
CodeWithYu
8 months agoApache Flink For Sales Analytics - End to End Data Engineering
7 -
58:56
CodeWithYu
9 months agoJapan Visa Analysis: Azure Data End to End Data Engineering
11