Quickstart: Google Cloud Pipeline Components
Stay organized with collections
Save and categorize content based on your preferences.
This quickstart guides you through the installation of the Google Cloud Pipeline Components (GCPC) SDK.
Install latest release
Use the following command to install the Google Cloud SDK from the Python Package Index (PyPI):
pip install --upgrade google-cloud-pipeline-components
Import a prebuilt component by using the Google Cloud SDK
After you install the Google Cloud SDK, you can use it to import a prebuilt component.
For SDK reference information for supported components, see the Google Cloud SDK documentation.
For example, you can use the following code to import and use the Dataflow component in a pipeline.
fromgoogle_cloud_pipeline_components.v1.dataflowimport DataflowPythonJobOp
fromkfpimport dsl
@dsl.pipeline(
name=PIPELINE_NAME,
description='Dataflow launch python pipeline'
)
defpipeline(
python_file_path:str = 'gs://ml-pipeline-playground/samples/dataflow/wc/wc.py',
project_id:str = PROJECT_ID,
location:str = LOCATION,
staging_dir:str = PIPELINE_ROOT,
requirements_file_path:str = 'gs://ml-pipeline-playground/samples/dataflow/wc/requirements.txt',
):
dataflow_python_op = DataflowPythonJobOp(
project=project_id,
location=location,
python_module_path=python_file_path,
temp_location = staging_dir,
requirements_file_path = requirements_file_path,
args = ['--output', OUTPUT_FILE],
)
What's next
- Read the Introduction to Google Cloud Pipeline Components.
- See all tutorials that use the
google_cloud_pipeline_componentsSDK. - Get started with Dataflow components.