Uploading user data

The basic enlighten web distribution is empty. Unless your installation has been configured by a system administrator there are initially datasets available. There are several ways that a user may load their data in to enlighten for further analysis.

Upload from pandas

Enlighten web understands pandas data frames (2 dimensional tabular data used in python). If enabled you can upload pandas dataframes through the REST interface.

For this example we will assume that your enlighten instance is running on port 8080 at IP 192.168.99.100. This is a likely scenario if you started enlighten from a docker image. Otherwise use the IP and port number of the instance that you are using. Initially your enlighten web is probably empty of data and looks something similar to the screenshot below:

_images/upload1.png

The upload REST interface supports several methods, in this case we use the msgpack method and demonstrate how to use this from a python code snippet:

import requests
import pandas as pd
data = {'state': ['Ohio', 'Ohio', 'Ohio', 'Nevada', 'Nevada', 'Nevada' ],
         'year': [ 2000,   2001,   2002,   2001,     2002,     2003],
         'pop' : [ 1.5,    1.7,    3.6,    2.4,      2.9,      3.2]}
frame = pd.DataFrame(data)
requests.post("http://192.168.99.100:8080/rest/upload/msgpack/pops", files=dict(file=frame.to_msgpack()))

In the example msgpack is the method used to transfer the data, and pops is the name of the dataset we are creating in our enlighten instance. The result is shown in the screenshot below:

_images/upload2.png

Upload from HDF

In a similar way to pandas upload we may use the HDF format for uploading data

import requests
import pandas as pd
data = {'state': ['Ohio', 'Ohio', 'Ohio', 'Nevada', 'Nevada', 'Nevada' ],
         'year': [ 2000,   2001,   2002,   2001,     2002,     2003],
         'pop' : [ 1.5,    1.7,    3.6,    2.4,      2.9,      3.2]}
frame = pd.DataFrame(data)
store = pd.HDFStore("pops",mode='a',driver='H5FD_CORE',driver_core_backing_store=0)
store["popshdf"] = frame
requests.post("http://192.168.99.100:8080/rest/upload/hdf5/popshdf", files=dict(file=store._handle.get_file_image()))