Add Item and Publish Service with ArcREST or ArcPy

Question

I would like to have an automated process which does the following:

1. Adds a CSV to our portaL/AGOL organization account.
2. Publishes the item as a hosted feature service.
3. Consumes hosted feature back unto EGIS environment.

I have checked out ArcREST as a possible solution, however, one hang up that I can imagine is the recurring ID number that will change as the added CSV is updated on a daily basis via task scheduler.

Is this solution possible with ArcREST?

I will post my code sample when I return to my desk.

I have used a piece of your code Kevin in addition to the addItem method from ArcREST. My item adds, and my token and item ID print, however, when I try to pass the itemID to the function below it does not publish and no errors are returned.

import arcrest
import json
import requests
import urllib
import urllib2
if __name__ == "__main__":
 username = "myusername"
 password = "mypassword"
 portalId = "myportalnumber"
 url = "http://myorg.maps.arcgis.com/"
 thumbnail_url = ''
 file = "myflie.csv"
 securityHandler = arcrest.AGOLTokenSecurityHandler(username,
 password)
 # Create the administration connection
 #
 admin = arcrest.manageorg.Administration(url, securityHandler)
 # Connect to the portal
 #
 portal = admin.portals(portalId)
 # Access the content properties to add the item
 #
 content = admin.content
 # Provide the item parameters
 #
 itemParams = arcrest.manageorg.ItemParameter()
 itemParams.thumbnailurl = thumbnail_url
 itemParams.title = "CSVTest"
 itemParams.type = "CSV"
 itemParams.tags = "test"
 # Enter in the username you wish to load the item to
 #
 usercontent = content.usercontent(username=username)
 # print usercontent.addItem(filePath=file,
 # itemParameters=itemParams, overwrite=True)
 agolID= usercontent.addItem(itemParameters=itemParams)['id']
 print agolID
 itemId = agolID
 tags = "Demo, Publishing"
 securityHandler = arcrest.AGOLTokenSecurityHandler(username,
 password)
### Generate Token ###
gtUrl = 'https://www.arcgis.com/sharing/rest/generateToken'
gtValues = {'username' : 'myuser',
'password' : 'mypass',
'referer' : 'http://www.arcgis.com',
'f' : 'json' }
gtData = urllib.urlencode(gtValues)
gtRequest = urllib2.Request(gtUrl, gtData)
gtResponse = urllib2.urlopen(gtRequest)
gtJson = json.load(gtResponse)
token = gtJson['token']
print token
def PublishService(Short, agolID, username, token, itemName, publishParams, XField, YField):
 ''' Publishes the input itemID (uploaded CSV) this is using
 a post request from urllib, the JSON input can be generated by
 the analyze function. Item name, X and Y fields are specified'''
 publishURL = 'http://myOrg.maps.arcgis.com/sharing/rest/content/users/myUser/publish'.format(Short, username)
 publishParams['CSVTest'] = itemName
 publishParams['locationType'] = 'coordinates'
 publishParams['Y_COR'] = YField
 publishParams['X_COR'] = XField
 query_dict = {
 'itemID': agolID,
 'filetype': 'csv',
 'f': 'json',
 'token': token,
 'publishParameters':publishParams}
 query_dict['publishParameters'] = json.dumps(query_dict['publishParameters'], sort_keys=False)
 request = requests.post(publishURL, data=query_dict)
 print request.json()`

Answer 1

I just uploaded a script uploads a CSV and publishes as a github-gist. You may want to change the input as right now its setup to run at command line and enter parameters manually when you execute.

The credit goes to a co-worker of mine who shared this with me. I haven't extensively tested it, but it worked for the case I was trying. Note that it does rely on the requests module. You'll need to make sure you've downloaded and installed that.

Also note the script isn't setup to re-publish or update/overwrite. I haven't done any work with re-publishing of a CSV file, but my best guess is you'll need to:

• delete the CSV from the day before (unless you can overwrite it, then this isn't necessary)
• upload the new CSV (that is in the gist)
• Call publish on the new CSV (that is in the gist)
• Set overwrite:true in publish. (that is NOT in the gist). See the doc on publish params here
• You should maintain the original ID because of the overwrite (I haven't tested this to verify)

I should also note, because you asked in your question, there are no true arcpy functions to upload a CSV to Portal. There are functions in ArcGIS Pro to create webmaps or weblayers and the UploadService Defintion tool, but that requires features in a map. Not a stand alone CSV.

• Geoffrey West: @KHimba please see updates
• MigueL: If you're looking for load CSV files using ARCGIS REST API see this doc : medium.com/@man9r3/…