Covered In This Guide:
- Prerequisites
- Submitting data - Simple Example
- [Optional] Submit numeric data with a different unit
- Submitting Data for a Log with a Resource Field
- Error Handling
💡The API Documentation can be found here.
Prerequisites
- You have an API key and have verified that you can successfully make a request (/ping).
- You know the structure of your source data (what you want to go into Mapistry) and how to access it.
- The Log for which you are setting up automated data submission exists in Mapistry (was built by your Mapistry CSM or organization admin) and is published.
Submitting Data - Simple Example
Submitting an entry with a numeric and text field.
💡 Please note that this is a setup for an example site and example Log - you will see different results from the endpoints depending on your own setup.
1. Identify the site
https://api.mapistry.com/public/api/v2/sites
Use the /sites endpoint to get a list of all sites your API user has access to.
The API user making the request here belongs to a few sites. For this example, I am looking to submit data to (Mapistry Demo) - Sam’s CA Site. I find that site ID in the response and save the site ID for future steps.
const DEMO_SITE_ID = 'd68bcb49-545c-4166-8b54-d4e213c583fc';
2. Identify the Log
https://api.mapistry.com/public/api/v2/edp/sites/{siteId}/logs
With DEMO_SITE_ID from step 1, retrieve the list of Logs available on that site.
(Mapistry Demo) - Sam’s CA Site has two Logs on it, and we want Simple Log. Again, we’re going to take that ID and save it for future steps.
const SIMPLE_LOG_ID = '0cf6304c-8553-4176-a2f8-c11ebd85b2b4';
3. Retrieve the Log configuration
https://api.mapistry.com/public/api/v2/edp/sites/{siteId}/logs/{logId}
With the DEMO_SITE_ID from step 1, and the SIMPLE_LOG_ID from step 2, we can retrieve more details about Simple Log – specifically, which pieces of data (fields) this Log expects.
In this response, we can see that there are two fields on Simple Log – Quantity Used (a number in gallons) and Notes (text). The Quantity Used field is required, while the Notes one is not - this depends on how your log was set up.
To submit data, we will need to know the IDs of these fields, so
const QUANTITY_USED_FIELD_ID = '63ed3ab1-5a6e-449f-aef5-c9de7fe024bf';
const NOTES_FIELD_ID = 'bf9d7bc5-d036-407d-861e-bed96f1113a1';
4. Submit Log entry via POST request
https://api.mapistry.com/public/api/v2/edp/sites/{siteId}/logs/{logId}/entries
We now have all of the information we need to submit EDP data to the API. Let’s put together a POST request and submit some sample data!
We receive a response of 200 with an array containing the single entry that was successfully submitted.
Here, we submitted data for a single entry, but you can submit multiple entries at once, up to 200 per request, by adding to the array of log entries in the request body .
5. Build a script to pull source data and push into Mapistry
This is a sample of a NodeJS script using axios. You can use whatever language and libraries you prefer, as long as you can make secure HTTP requests.
The purpose of this script is to outline the typical structure that is needed to support a data submission use case – retrieving source data, translating it into the form Mapistry expects, and POSTing it to Mapistry’s API. It is not a complete script, and is missing things like logging and implementation of error handling logic for the various error responses you may receive.
Note that the script uses the IDs retrieved in earlier steps to know what endpoint to send the request to and what the body of that request should look like.
const QUANTITY_USED_FIELD_ID = '63ed3ab1-5a6e-449f-aef5-c9de7fe024bf';
const NOTES_FIELD_ID = 'bf9d7bc5-d036-407d-861e-bed96f1113a1';
function getSourceData() {
// this data could be retrieved from an internal database, a different API, a csv or other file, etc - wherever your data lives
// for simplicity of this script and for better demonstration, the example data is hardcoded here
return [
{
date: new Date('2024-02-01 14:00'),
quantityInGallons: 12.5,
notes: null,
},
{
date: new Date('2024-02-02 11:00'),
quantityInGallons: 8.3,
notes: null,
},
{
date: new Date('2024-02-03 11:00'),
quantityInGallons: 20.5,
notes: 'Leak identified',
},
];
}
function translateDataPointForMapistry(sourceDataPoint) {
// map the internal data point into Log entries Mapistry will understand
return {
logDate: format(sourceDataPoint.date, "yyyy-MM-dd'T'HH:mm"),
fieldValues: {
[QUANTITY_USED_FIELD_ID]: {
value: sourceDataPoint.quantityInGallons,
units: 'gal',
},
[NOTES_FIELD_ID]: { value: sourceDataPoint.notes },
},
};
}
function handlePostSuccess(response) {
// could do some logging so you have a record of the successful data submission
}
function handlePostError(response, requestBody) {
// at a minimum, you'll want to log that the error occurred and data was not successfully submitted, and potentially try again
// depending on the response code, there are different things you may be able to do
if (response.code === 429) {
// too many requests - wait some time and then retry the submission
// use `e.response.headers['Retry-After']` to know how long to sleep before retrying the request
}
if (response.code === 400 || response.code === 422) {
// malformed request or validation error
// something is wrong with at least one of the entries submitted
// the error body likely includes details about what exactly needs fixing
// this almost certainly will require intervention, so log / alert someone who can review the error and fix things
}
// see API documentation for more error types
}
async function postData(sourceData) {
const http = axios.create();
const requestBody = sourceData.map(translateDataPointForMapistry);
// the post endpoint limits you to submitted 200 entries at a time, so if you have more than that you will need to break them into batches
try {
const response = await http.post(
`${API_URL}/edp/sites/${DEMO_SITE_ID}/logs/${SIMPLE_LOG_ID}/entries`,
requestBody,
{
headers: {
// MUST be included or you will receive 401 response
Authorization: `Bearer ${SECRET_API_KEY}`,
},
},
);
handlePostSuccess(response);
} catch (e) {
handlePostError(e.response, requestBody);
}
}
postData(getSourceData());
[Optional] Submit numeric data with a different unit
In the above example, we submit the Quantity Used value in gallons, which is the unit that the field was configured with during setup. Mapistry is unit-aware, which means that you are able to submit data for a field in a different, compatible unit than the field was configured with.
Let’s say that one of the sources for this Log measures the Quantity Used in cubic meters. You are able to send that data without having to do unit conversion to gallons. Here are the additional steps you would take to do so.
If you look back to the …/logs/{logId} reponse, we can see that the unit configured for the field is gal. That is the unit we passed in steps 4 and 5 above. To find out what other options you have for units on values for this field you can use the /related-units endpoint.
There are a number of available volume units, but if we scroll through the response we find that we can use ”m^3”. Each unit in the response includes a human-friendly label (though most of the IDs are self-explanatory).
The last step is to use that unit’s ID in the POST request with your entry submission. Here is a new sample POST body, this time submitting the data for Quantity Used in cubic meters.
[
{
"logDate": "2024-02-21T00:00",
"fieldValues": {
"63ed3ab1-5a6e-449f-aef5-c9de7fe024bf": {
"value": 3.5,
"units": "m^3"
},
"bf9d7bc5-d036-407d-861e-bed96f1113a1": {
"value": "This time in cubic meters."
}
}
}
]
Submitting Data for a Log with a Resource field
For this example, we will be expanding on what was done in the previous section to support a slightly more complex use case – submitting data for a Log that has a Resource field. See the Mapistry product guide for a background on Resources.
The first two steps are the same as the previous example, but for completeness they are included here.
1. Identify the Site
https://api.mapistry.com/public/api/v2/sites
Identical to step 1 from “Submitting Data - Simple Example” above.
const DEMO_SITE_ID = 'd68bcb49-545c-4166-8b54-d4e213c583fc';
2. Identify the Log
https://api.mapistry.com/public/api/v2/edp/sites/{siteId}/logs
Same process as step 2 from “Submitting Data - Simple Example” above, but this time we are going to use the Generator Usage Log. So we select that ID from the list in the response, and continue.
const GENERATOR_USAGE_LOG_ID = 'e8507ca4-e175-4aaf-91b0-4d6e6b556067';
3. Retrieve the Log configuration
https://api.mapistry.com/public/api/v2/edp/sites/{siteId}/logs/{logId}
Again, this step is analogous to the “Submitting Data - Simple Example” section’s step 3, but the results are different because we’re retrieving a different Log’s configuration.
In the response, we see two fields that we can submit for the Generator Usage Log – Throughput (a number in tons) and Generator, which is a Resource field. Like the first example, we save the field IDs for later when we will submit data.
const THROUGHPUT_FIELD_ID = '79b621e4-f509-4913-b0ce-96dd133730de';
const GENERATOR_FIELD_ID = '48b2767f-6aa8-42e2-84f2-471bafcf12cb';
The third ID we need from this response is in the third box of the screenshot. We’ll come back to this in the next step.
const GENERATOR_RESOURCE_TYPE_ID = '10470b20-0b61-4d7d-a7a8-b2d3e88e3f1c';
4. Retrieve the list of Resources
https://api.mapistry.com/public/api/v2/edp/sites/{siteId}/resourceTypes/{resourceTypeId}/resources
For the numeric and text fields that we previously came across, what we could submit as values for each of them was straightforward. Text fields accept strings, and numeric fields accept numbers (with their unit). Fields of type “resource” are more specific, and require that the submitted value be one of the options from the list of available Resources of a given Resource type.
To get this list of options, we use the Resources endpoint.
From the response, we now have the IDs of the options (“Gen 1” and “Gen 2”) we can submit to the Generator field. Gen 1 and Gen 2 here are referencing actual generators that all belong to the same Generator Resource type and exist on our demo site.
const GEN_1_RESOURCE_ID = 'bdcd614a-489d-480c-9502-96f950a58932';
const GEN_2_RESOURCE_ID = '1d95e69a-27aa-4ee2-bd79-18a65b948e38';
5. Submit Log entries via POST request
https://api.mapistry.com/public/api/v2/edp/sites/{siteId}/logs/{logId}/entries
Data submission time. Let’s say we want to log usage of each generator: 10 tons completed using “Gen 1” and 5 tons completed using “Gen 2”.
We received a response of 200 with an array containing the two entries that were successfully submitted!
From here, you can build a script that will pull your source data, map it into the shape expected by the POST endpoint, and submit the data to the API. With a Resource field on your Log, you will have more constants in your code, but the concept from step 5 of “Submitting Data - Simple Example” stays the same.
Error Handling
Refer to the “Responses” documentation on each of the endpoints of the API documentation for details on the various error responses you may encounter while working with the Mapistry public API. This section will go into detail about a couple common error cases, and how you might want to handle them.
Access Error (403)
Your user doesn’t have access to the site for which you are trying to submit data. You will need to contact a site admin or Mapistry Customer Success.
Validation Errors (422)
Validation errors almost certainly require review by someone who is familiar with the data being submitted, so that they can sort through what the problem may be. Sometimes a validation error happens because the data itself is invalid, and sometimes a validation error occurs just because it wasn’t formatted in the way Mapistry expects it to be. The error body will include more information about what exactly went wrong. In the event of a 422 error response, you will want to log that the error occurred (including its details) and be sure that those error logs are getting reviewed at appropriate intervals.
It’s important to note that if any of the log entries in a single request have validation errors, then none of them will be saved to Mapistry. If you’d like to submit just the valid entries, then you could look at the error response to determine which log entries are invalid, remove those from the request body, and retry saving the “good” entries.
Throttling (429)
The Mapistry public API has rate limiting built into it to ensure a performant experience for all users. Public API users are restricted to submitting 20 POST requests for Log entry data per minute.
The good news is that you should be able to completely handle any throttling errors in your script, without need for human intervention! On receiving a 429 response, you can have your script wait / sleep until your available requests refill (using the Retry-After header to determine how long to wait) and then retry the same request.
Below is what this handling might look like if we added it onto the scripts from earlier in this document (for a production-ready script, you’ll very likely want to have in place a maximum number of retries).
async function handlePostError(response, requestBody) {
if (response.status === 429) {
const secondsToWait = parseFloat(response.headers['Retry-After']);
await sleep(secondsToWait * 1000); // assuming the sleep function expects ms
await postData(requestBody); // attempt the same POST request again
}
// other error response handling...
}
Internal Server Error (500)
Every once in a while, the Mapistry server may have a short outage or other issue. In situations where you receive a 500 error code, we recommend that you wait for a little while (something like the sleep in the Throttling example code) and then retry. If the error keeps happening, please contact us with details about your request – which endpoint, the time at which you made the request, and any other information you think might be useful.
End of Article
Support
If you have any questions or need further assistance, please e-mail help@mapistry.com or call (510) 313-1422.