Retreiving datas from the server

This page has been updated on the 04/01/2010.

Retreiving datas in java

The aim of this document is to help you start using the CloudConnect apis.
With this tutorial, you will learn to retrieve tracking datas from your devices and then save it to a file. Tracking datas are described by what we call tracking records.

Tracking Records

A tracking is described by:

  • unitid : a unique identifier of your devices in the instance of the CloudConnect. To find out what is the unitid if your device you can go on the search tool of the resource unit and enter the modid of your device.
  • recordtime: record time of the tracking records
  • latitude, longitude: location of the tracking record
  • id (fieldid): identifier of the data recorded (for example: the id for speed is 8)
  • value: recorded value

En example of Tracking record:

[{"tracking_record":
   {"latitude": 0,
   "recordtime": "2009-04-10T13:18:59Z",
    "unitid": 4,
    "id": 1,
    "value": "%B9q",
    "longitude": 0}]

In order to get tracking records there will be 2 type of requests. The first one will be to login, the others will fetch the data.

Login

As explained in the documentation, to login, you must use a POST request to session resource with your credentials. If the request is correct, you will receive an http 302 redirect that will you get to the index of units and a set-cookie header. This cookie represent your credential and you must send with all of your requests.

// we use sessions resource
URL url = new URL("http://www.g8teway.com/api/v2/sessions.json");
//create the connection to java API
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
// use HTTP POST
connection.setRequestMethod("POST");
connection.setDoOutput(true);
connection.connect();
OutputStreamWriter writer = new OutputStreamWriter(connection.getOutputStream());
//write our credential in the socket
String parameters = "username=" + YOURLOGIN + "&password=" + YOURPASSWORD + "&client=" + YOURCLIENT + "";
writer.write(parameters);
writer.flush();
//close it to get the cookie
connection.disconnect();
// save the cookie
String cookie = connection.getHeaderField("Set-Cookie");

Get the tracking records

The next thing you want to do with your brand new cookie is to get some tracking records. To get all, there is 2 concept to understand.
First the number of tracking records you can get in 1 request is limited to 1500. To get more, you will need to issue a new request add use the start parameter or the id_min parameter. The second limitation is on the number of requests by seconds. Currently the limit is set to 1 request by second but this value can change in future. So if you want to get more than 1500 tracking records you will need to use both pages of tracking records and some sleep instruction in your code. Here is the code to get 1500 tracking records max in 1 request.

//configure the filters
String parameters = "?ret=id,unitid,lat,lng,fieldid,data"
// we use tracking_records as resource with format in json
URL url = new URL("http://www.g8teway.com/api/v2/tracking_records.json" + parameters);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
// Important, this is your credential to the app
connection.setRequestProperty("Cookie", cookie);
// HTTP Get method
connection.setRequestMethod("GET");
connection.setDoOutput(false);
connection.connect();
BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
// decode the json data
JSONArray res = new JSONArray(reader.readLine());

Paging and time limitation

OK, so we have tracking records, but we do not have all tracking records. And moreover we do not check if new tracking records have been sent to the CloudConnect. So we will now make multiple request to the api to be sure we have everything. The basic idea will be first to ask quickly the api to give me all the pages of tracking records I want, and then check less frequently if there are new tracking records available. For the first part I will use the parameter id_min of the api which basically tell the api: “hey do not give me anything before this id, I already have it”. You just have to keep it updated each time you receive new tracking records. Between each of these requests do not forget to use a sleep instruction of at least 1 second, otherwise the server might stop responding to your requests given you have exceed your request quota. Here is some code to do that.

boolean finished = false;
JSONArray res = new JSONArray();
while (!finished) {
  JSONArray newValues = requestTrackingRecords(); //this function is from the previous part
  for (int i = 0; i < newValues.length(); i++) {
    res.put(newValues.get(i)); //append new values to the res
  }
  //have we finished ?
  if (newValues.length() == 0) {
    finished = true;
  }
  else {
    //no more than 1 requests every 10 seconds
    Thread.sleep(10 * 1000);
  }
}

This will put in res all the tracking records. The only things to do is to put this code in some kind of loop with another sleep instruction and save the res as you like.

Put it together

You now have all you need to do a little Api class that will do the job. I have also add some constants to parameter the frequency, credentials, …

import java.io.*;
import java.net.*;
import java.util.Date;

//require java json to decode (found at http://www.json.org/java)
import org.json.*;

public class Api {

// user constants- change it with your values
 public static final String BASEURL = "http://www.g8teway.com/api/v2/";
  private static final String LOGIN = "";
  private static final String PASSWORD = "";
 private static final String CLIENT = "";
  // this is the list of tracking records you want (for instance 14 is ignition and 24 odometer)
  private static final String FIELDIDS = "14,24";
  // this is the list of units you want tracking records from.
  private static final String UNITIDS = "3,5";
  // this is the field you want to have in the response from api private static final String RET = "id,unitid,lat,lng,fieldid,data";

// the name of the file that will be created private static final String FILENAME = "out.txt";

// APIs constants
  // frequency used to get all new tracking records by packet of 500 private static final int BURSTFREQ = 10 * 1000; 10 seconds in milliseconds; frequency used to get new tracking data
  //private static final int NORMALFREQ = 2 * 60 * 1000; // 2 minutes in milliseconds
  // page size private static final int PAGESIZE = 500; // 2 minutes in milliseconds

// class attributes
  private HttpURLConnection connection = null;
  private String cookie = "";
  // this will enable to request only the new tracking records private int lastId = 0;

// used to log in the user
  /**
   * Enable the user to login using credentials in constants at the beginning
  * of this file
   *
   * param login
   *            your login
   * param password
   *            your password
   * param client
   *            your instance name
   * throws Exception
  */
 public void login(String login, String password, String client)
      throws Exception {

// we use sessions resource
    URL url = new URL(BASEURL + "sessions.json");
    connection = (HttpURLConnection) url.openConnection();
    // use HTTP POST
    connection.setRequestMethod("POST");
    connection.setDoOutput(true);
    connection.connect();
    OutputStreamWriter writer = new OutputStreamWriter(connection.getOutputStream());
   String parameters = "username=" + login + "&password=" + password + "&client=" + client + "";
    writer.write(parameters);
    writer.flush();
    connection.disconnect();
    // save the cookie
    cookie = connection.getHeaderField("Set-Cookie");

}// end login

// used to get the next tracking records
  /**
   * This will make 1 request to apis to get the next tracking records page
   * (500 tracking records max by page). Please note that this update the
   * lastId attributes in order to request only new tracking records on the
   * next request
   * 
   * return A JSONArray representing the result of the request
   * throws Exception
 */
 public JSONArray requestTrackingRecords() throws Exception {
    System.out.println("new request : lastid = " + lastId + " at " + new Date());
    // the parameters of the request (including the id_min)
    String parameters = "?ret=" + RET + "&fieldids=" + FIELDIDS + "&unitids=" + UNITIDS + "&limit=" + PAGESIZE + "&id_min=" + (lastId + 1);
    // we use tracking_records as resource with format in json
    URL url = new URL(BASEURL + "tracking_records.json" + parameters);
   connection = (HttpURLConnection) url.openConnection();
    // Important, this is your credential to the app
    connection.setRequestProperty("Cookie", cookie);
    // HTTP Get method
    connection.setRequestMethod("GET");
    connection.setDoOutput(false);
    connection.connect();
    BufferedReader reader = new BufferedReader(new InputStreamReader( connection.getInputStream()));
    // decode the json data
    JSONArray res = new JSONArray(reader.readLine());
    // update last_id attributes if needed
    if (res.length() > 0) {
     lastId = res.getJSONObject(res.length() - 1).getJSONObject(
          "tracking_record").getInt("id");
    }
    return res;

}// end requestTrackingRecords

/**
   * This will request the different pages of new tracking records using
   * BURSTFREQ constant.
   * 
   * return A JSONArray representing all the new tracking records
   * throws Exception
   */
 public JSONArray requestAllNewTrackingRecords() throws Exception {
   boolean finished = false;
    JSONArray res = new JSONArray();
    while (!finished) {
      JSONArray newValues = requestTrackingRecords();
      for (int i = 0; i < newValues.length(); i++) {
        res.put(newValues.get(i));
      }
      if (newValues.length() == 0) {
        finished = true;
     } else {
        Thread.sleep(BURSTFREQ);
      }
   }
    return res;
  }

/**
   * Main for demo purpouse
   */
  public static void main(String[] args) throws Exception {

Api api = new Api();
    Writer output = null;
    try {
     // first login
     System.out.println("Login");
      api.login(LOGIN, PASSWORD, CLIENT);
      System.out.println("Cookie is " + api.cookie);

File file = new File(FILENAME);
      output = new BufferedWriter(new FileWriter(file));

// main loop
      while (true) {
        JSONArray json = api.requestAllNewTrackingRecords();
       for (int i = 0; i < json.length(); i++) {
          // store your data where you want.
          // here it will be in a file, 1 tracking records by line
         output.write(json.getJSONObject(i) + "\n");
        }
        output.flush();
        Thread.sleep(NORMALFREQ);
      }// end main loop

} catch (Exception e) {
      // error management
      System.out.println("Error");
      e.printStackTrace();
   } finally {
      // close the socket
     if (api.connection != null) {
        // System.out.println("Disconnect");
        api.connection.disconnect();
      }
      // close the file
      if (output != null) {
        output.close();
      }
    }// end try

}// end main
}