Connect from Compute Engine

This guide gives instructions on creating a single Compute Engine client and connecting it to your Parallelstore instance.

To create and connect from multiple Compute Engine clients, you can follow the instructions in Connect from Compute Engine: multiple clients.

For better performance, client Compute Engine VMs should be created in the same zone as the Parallelstore instance.

Required permissions

You must have the following IAM role in order to create a Compute Engine VM:

• Compute Instance Admin (v1) (roles/compute.instanceAdmin.v1). For more information, refer to the Compute Engine documentation.

Create a Compute Engine VM

Follow the instructions to create a Compute Engine VM using one of the following images:

• HPC Rocky Linux 8
• Rocky Linux 9 Optimized
• Red Hat Enterprise Linux (RHEL) 9
• Ubuntu 22.04
• Debian 12

You can choose any machine type and boot disk. We recommend at least a c2-standard-4 machine type; for higher client performance, increase the number of vCPUs to increase the network throughput. For example, a c3-standard-176 with Tier 1 networking provides 200Gbps of egress bandwidth.

gcloudcomputeinstancescreateVM_NAME\
--project=PROJECT_ID\
--zone=LOCATION\
--machine-type=c2d-standard-112\
--network-interface=stack-type=IPV4_ONLY,subnet=NETWORK_NAME,nic-type=GVNIC\
--network-performance-configs=total-egress-bandwidth-tier=TIER_1\
--create-disk=auto-delete=yes,boot=yes,device-name=VM_NAME,\
image=projects/cloud-hpc-image-public/global/images/hpc-rocky-linux-8-v20240126,\
mode=rw,size=100,type=pd-balanced

gcloudcomputeinstancescreateVM_NAME\
--project=PROJECT_ID\
--zone=LOCATION\
--machine-type=c2d-standard-112\
--network-interface=stack-type=IPV4_ONLY,subnet=NETWORK_NAME,nic-type=GVNIC\
--network-performance-configs=total-egress-bandwidth-tier=TIER_1\
--create-disk=auto-delete=yes,boot=yes,device-name=VM_NAME,\
image=projects/rocky-linux-cloud/global/images/rocky-linux-9-optimized-gcp-v20241112,\
mode=rw,size=100,type=pd-balanced

gcloudcomputeinstancescreateVM_NAME\
--project=PROJECT_ID\
--zone=LOCATION\
--machine-type=c2d-standard-112\
--network-interface=stack-type=IPV4_ONLY,subnet=NETWORK_NAME,nic-type=GVNIC\
--network-performance-configs=total-egress-bandwidth-tier=TIER_1\
--create-disk=auto-delete=yes,boot=yes,device-name=VM_NAME,\
image=projects/rhel-cloud/global/images/rhel-9-v20241112,\
mode=rw,size=100,type=pd-balanced

gcloudcomputeinstancescreateVM_NAME\
--project=PROJECT_ID\
--zone=LOCATION\
--machine-type=c2d-standard-112\
--network-interface=stack-type=IPV4_ONLY,subnet=NETWORK_NAME,nic-type=GVNIC\
--network-performance-configs=total-egress-bandwidth-tier=TIER_1\
--create-disk=auto-delete=yes,boot=yes,device-name=VM_NAME,\
image=projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20240927,\
mode=rw,size=100,type=pd-balanced

gcloudcomputeinstancescreateVM_NAME\
--project=PROJECT_ID\
--zone=LOCATION\
--machine-type=c2d-standard-112\
--network-interface=stack-type=IPV4_ONLY,subnet=NETWORK_NAME,nic-type=GVNIC\
--network-performance-configs=total-egress-bandwidth-tier=TIER_1\
--create-disk=auto-delete=yes,boot=yes,device-name=VM_NAME,\
image=projects/debian-cloud/global/images/debian-12-bookworm-v20240415,\
mode=rw,size=100,type=pd-balanced

For more information about available options, see the Compute Engine documentation.

SSH to the client VM

gcloudcomputefirewall-rulescreateFIREWALL_RULE_NAME\
--allow=tcp:22\
--network=NETWORK_NAME\
--source-ranges=0.0.0.0/0\
--project=PROJECT_ID

gcloudcomputesshVM_NAME--zone=ZONE--project=PROJECT_ID

Install the DAOS client library

The DAOS client library provides a POSIX-like interface to the Parallelstore data layer. The software runs as an agent on your client machines and must be installed and run before you can access your data.

Increase the open files limit (Ubuntu only)

For VMs running Ubuntu 22.04, you must increase the open files limit to 131072 to support dfuse and the interception library.

If you choose not to use the interception library, you can alternatively run ulimit -n 131072 immediately before starting dfuse.

To increase the open files limit from 1024, run the following commands on each VM.

sudotee-a/etc/security/limits.conf<<EOF
*softnofile131072
*hardnofile131072
EOF

Then, reboot:

sudoreboot

SSH to the client VM again once it finishes rebooting.

Update the DAOS agent configuration

Update /etc/daos/daos_agent.yml as follows:

1. Uncomment and update access_points with the accessPoints IP addresses from the Parallelstore instance properties. For example: access_points: ['172.21.95.2', '172.21.95.4', '172.21.95.5'].

   To print the access points in the correct format to copy and paste, run the following command:

   echoaccess_points\:$(gcloudbetaparallelstoreinstancesdescribe\
   INSTANCE_ID--locationLOCATION--projectPROJECT_ID\
   --format"value[delimiter=', '](format("{0}", accessPoints))")
   

2. Uncomment the following two lines. Indentation matters so make sure to keep the spaces in front of allow_insecure:

   # transport_config:
   # allow_insecure: false
   

3. Change the value of allow_insecure to true as certificates are not supported.

   transport_config:
   allow_insecure:true
   

4. Specify the network interface that provides connectivity to the Parallelstore instance. The interface is often eth0, ens4, or enp0s3, but might be different depending on your network configuration. You can use the route command to show your VM's default gateway; the interface to specify is usually the one sharing a subnet with the gateway.

   First, list all available network interfaces:

   ipa
   

   The output is similar to the following:

   1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    valid_lft forever preferred_lft forever
   2: eth0@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc noqueue state UP group default
    link/ether e4:9x:3f:x7:dx:f7 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.88.0.3/16 brd 10.88.255.255 scope global eth0
    valid_lft forever preferred_lft forever
   3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1460 qdisc noqueue state DOWN group default
    link/ether 02:4x:6y:1z:84:45 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
    valid_lft forever preferred_lft forever
   

   Run route to display the routing table:

   route
   

   The output is similar to the following:

   Kernel IP routing table
   Destination Gateway Genmask Flags Metric Ref Use Iface
   default 10.88.0.1 0.0.0.0 UG 0 0 0 eth0
   10.88.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
   172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
   

   In the example, the default gateway is 10.88.0.1, and it's shared by eth0, so specify eth0 as the interface to use.

   Edit /etc/daos/daos_agent.yml. Uncomment include_fabric_ifaces and update the value:

   include_fabric_ifaces: ["eth0"]
   

   Save and close the file.

Start the DAOS agent

sudosystemctlstartdaos_agent.service

systemctlstatusdaos_agent.service

sudosystemctlstartdaos_agent.service

systemctlstatusdaos_agent.service

sudosystemctlstartdaos_agent.service

systemctlstatusdaos_agent.service

sudomkdir/var/run/daos_agent && \
sudodaos_agent-o/etc/daos/daos_agent.yml&

sudomkdir/var/run/daos_agent && \
sudodaos_agent-o/etc/daos/daos_agent.yml&

Set up logging

Set up local logging to aid with client-side debugging, if necessary:

exportD_LOG_MASK=INFO
exportD_LOG_FILE_APPEND_PID=1
rm-f/tmp/client.log.*
exportD_LOG_FILE=/tmp/client.log

Mount the instance using dfuse

Mount the Parallelstore instance using dfuse (DAOS FUSE).

1. Edit /etc/fuse.conf to add user_allow_other.

2. Specify the --multi-user option with dfuse:

   mkdir-p/tmp/parallelstore
   dfuse-m/tmp/parallelstore\
   --pooldefault-pool\
   --containerdefault-container\
   --disable-wb-cache\
   --thread-count=20\
   --eq-count=10\
   --multi-user
   

For help optimizing the values of --thread-count and --eq-count, see the Thread count and event queue count section of the Performance considerations page.

Access your Parallelstore instance

Your Parallelstore instance is now mounted to your Compute Engine VM at the path specified by the -m flag, and readable/writable using standard POSIX syntax, with some exceptions.

If you run df on the instance, the SIZE value is 1.5x the value specified with --capacity-gib. The amount of usable space is still --capacity-gib due to the nature of the erasure encoding used by Parallelstore. Every 2 bytes written uses 3 bytes from the perspective of df.

Unmount the instance

The Parallelstore instance can be unmounted using the following command:

sudoumount/tmp/parallelstore/

What's next

• Transfer data to Parallelstore from Cloud Storage