com.datasift.client.push.connectors

Class Http

java.lang.Object

com.datasift.client.push.connectors.BaseConnector<Http>

com.datasift.client.push.connectors.Http

All Implemented Interfaces:

PushConnector<Http>

public class Http
extends BaseConnector<Http>

Official docs

Author:

Courtney Robinson

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Http.AuthType
static class	Http.HttpFormat

Field Summary

Fields inherited from class com.datasift.client.push.connectors.BaseConnector

params, PREFIX, required

Constructor Summary

Constructors

Constructor and Description
Http()

Method Summary

Methods

Modifier and Type	Method and Description
Http	auth(String username, String password) Sets the authentication information that should be used for the connector
Http	authType(Http.AuthType auth) The authentication that you want DataSift to use when connecting to output_params.url: basic none If you choose basic authentication, you must supply output_params.auth.username and output_params.auth.password.
Http	compression(String format)
Http	deliveryFrequency(int freq) The minimum number of seconds you want DataSift to wait before sending data again: 10 (10 seconds) 30 (30 seconds) 60 (1 minute) 120 (2 minutes) 300 (5 minutes) In reality, a stream might not have data available after the wait.
Http	format(Http.HttpFormat format) The output format for your data: json_meta - The current default format, where each payload contains a full JSON document.
Http	gzip()
Http	maxSize(int max) The maximum amount of data that DataSift will send in a single batch: 102400 (100KB) 256000 (250KB) 512000 (500KB) 1048576 (1MB) 2097152 (2MB) 5242880 (5MB) 10485760 (10MB) 20971520 (20MB) 52428800 (50MB) 104857600 (100MB) 209715200 (200MB)
Http	method(io.netty.handler.codec.http.HttpMethod method) The verb that you want DataSift to use with the HTTP request: POST PUT
Prepared	parameters() Every push connector has a set of output parameters used to configure the destination.
Http	password(String password)
Http	url(String url) Any valid URL that you want DataSift to deliver to; for example: http://www.fromdatasift.com/destination/ For POST requests: DataSift uses the URL that you specify.
Http	useGzip(boolean trueOrFalse) Deprecated. use gzip() OR zlib() instead
Http	username(String username)
Http	verifySSL(boolean trueOrFalse) If you are using SSL to connect, this specifies whether the certificate should be verified.
Http	zlib()

Methods inherited from class com.datasift.client.push.connectors.BaseConnector

exists, fromMap, putAll, setParam, setup, type

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Http

public Http()

Method Detail

format

public Http format(Http.HttpFormat format)

The output format for your data:

json_meta - The current default format, where each payload contains a full JSON document. It contains metadata and an "interactions" property that has an array of interactions.

json_array - The payload is a full JSON document, but just has an array of interactions.

json_new_line - The payload is NOT a full JSON document. Each interaction is flattened and separated by a line break. If you omit this parameter or set it to json_meta, your output consists of JSON metadata followed by a JSON array of interactions (wrapped in square brackets and separated by commas). Take a look at our Sample Output for File-Based Connectors page. If you select json_array, DataSift omits the metadata and sends just the array of interactions. If you select json_new_line, DataSift omits the metadata and sends each interaction as a single JSON object.

Returns:

this

method

public Http method(io.netty.handler.codec.http.HttpMethod method)

The verb that you want DataSift to use with the HTTP request:

POST

PUT

Returns:

this

url

public Http url(String url)

Any valid URL that you want DataSift to deliver to; for example: http://www.fromdatasift.com/destination/

For POST requests: DataSift uses the URL that you specify.

For PUT requests: DataSift appends a filename to the URL. For example, suppose that you supply this URL: http://www.fromdatasift.com/destination/

Internally we append the filename in this format: DataSift--.json

When you hit the /push.create endpoint for the first time, we make a PUT request to http://www.fromdatasift.com/destination/DataSift-verify-31546216.json (where there isn't a subscription id created yet, so we add "verify" into the file name and 31546216 is the time of test).

When DataSift has data ready for delivery using a PUT request, it sends it to http://www.fromdatasift.com/destination/DataSift-abcdefghij1234579-31546216.json (where the abcdefghij1234579 is the subscription id and 31546216 is the time of delivery). Make sure that the URL is properly encoded, otherwise your push/create request will fail. This on-line tool can help you properly encode data you want to use in an HTTP request.

Returns:

this

deliveryFrequency

public Http deliveryFrequency(int freq)

The minimum number of seconds you want DataSift to wait before sending data again:

10 (10 seconds)

30 (30 seconds)

60 (1 minute)

120 (2 minutes)

300 (5 minutes)

In reality, a stream might not have data available after the wait. Typically this happens in streams that have very tight filtering constraints, so the wait time might be longer than you specify. If your system is capable of handling large amounts of incoming data, we recommend you use continuous delivery:

0 (continuous delivery)

Returns:

this

maxSize

public Http maxSize(int max)

The maximum amount of data that DataSift will send in a single batch: 102400 (100KB) 256000 (250KB) 512000 (500KB) 1048576 (1MB) 2097152 (2MB) 5242880 (5MB) 10485760 (10MB) 20971520 (20MB) 52428800 (50MB) 104857600 (100MB) 209715200 (200MB)

Returns:

this

authType

public Http authType(Http.AuthType auth)

The authentication that you want DataSift to use when connecting to output_params.url:

basic

none

If you choose basic authentication, you must supply output_params.auth.username and output_params.auth.password. If you specify "none" for authentication, or if you do not include this parameter, DataSift does not check for a username or password.

Returns:

this

verifySSL

public Http verifySSL(boolean trueOrFalse)

If you are using SSL to connect, this specifies whether the certificate should be verified. Useful when a client has a self-signed certificate for development. Possible values are: true

false

Returns:

this

useGzip

public Http useGzip(boolean trueOrFalse)

Deprecated. use gzip() OR zlib() instead

The compression setting that you want DataSift to use: true false If you set this parameter to true, DataSift compresses the data using the ZLIB compression standard with the compression level 6. We also add an additional entry to the header: Content-Encoding: gzip

Returns:

this

gzip

public Http gzip()

zlib

public Http zlib()

compression

public Http compression(String format)

auth

public Http auth(String username,
        String password)

Sets the authentication information that should be used for the connector

Parameters:

username - the username

password - the password

Returns:

this

username

public Http username(String username)

Parameters:

username - he username for authorization.

Returns:

this

password

public Http password(String password)

Parameters:

password - The password for authorization.

Returns:

this

parameters

public Prepared parameters()

Description copied from interface: PushConnector

Every push connector has a set of output parameters used to configure the destination. It is expected that this method will return a map of the form

output_params.variable-name ==> value

Each connector implementation should* do it's own verification of the map's contents

Specified by:

parameters in interface PushConnector<Http>

Overrides:

parameters in class BaseConnector<Http>