AWS Nodejs Athena Client
Description
Run multiple SQL like commands against Athena in order, results write to a stream.
Usage Example
AWS credentials are not required and may be provided via ~/.aws/config and ~/.aws/credentials files or via enviromental variables or vi AWS IAM permissions, etc.
The default output is JSON. This works well for SQL like statements list SELELCT but not for results from ATHENA only statements like 'MSCK REPAIR TABLE action'. To see proper results from those or any statement use the request.setFormatJson(false) method.
// EXAMPLE USAGE:// Requires node 7.6 or greater for native async/awaitconst Athena = ; // Create the Athena client and set the options.// See https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Athena.html.// For all other options.const CLIENT = Athena; // Wrap in an async function only if you want to keep calls asynchronous.// But note that if you do NOT make the calls asyncronously OR throttle the queries,// then AWS will throttle multiple syncronous Athena queries. { for let i=0; i<querieslength; i++ // Loop through the queries and await for each to finish. await CLIENT; } // A list of example queries to run.const sampleQueries = "CREATE DATABASE IF NOT EXISTS mytestschema;" "DROP TABLE IF EXISTS mytestschema.mytesttable;" "CREATE EXTERNAL TABLE IF NOT EXISTS mytestschema.mytesttable( `one` string, `two` string, `three` string) PARTITIONED BY ( year int, month int, day int ) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' WITH SERDEPROPERTIES ( 'serialization.format' = '1') LOCATION 's3://somebucket/some/path/' TBLPROPERTIES ('has_encrypted_data'='false');" "ALTER TABLE mytestschema.impression ADD IF NOT EXISTS PARTITION (year='2017',month='9',day='25',hour='20') location 's3://somebuckt/some/path/2017/09/25/20';" "SELECT * FROM mytestschema.mytesttable;"; // See the magic happen;
API
Athena = require("aws_athena_client")
Contains two parts the Athena client that can then take one or more Requests each which have a query and an optional output part.
Client = Athena.Client()
Returns an empty client, must call setOptions
Client.setOptions(<{aws options here}>)
See https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Athena.html. All the options that can be set are explained there. Typical values to pass in are:
// or some other region, // must be same region schema is in. region : 'us-east-1' accessKeyId: '<AwsAccessKeyHere>' secretAccessKey : '<AwsSecretKeyHere>'
Client.request(Request);
Does the actual call to AWS. See Request below.
Request = Athana.Request()
Gets an empty request
Request.setQuery
See https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Athena.html#startQueryExecution-property Typical query is:
QueryString: "SELECT * FROM mytestschema.mytesttable;" ResultConfiguration: OutputLocation: "s3://some_scratch_athena_bucket/"
Request.setRetryMax
A non zero positive number of the number of times to retry asking AWS should a failure occure. Defaults to 10.
Request.setRetryWait
A non zero positive number of the number of milli seconds to wait before asking the 1st time or any subsequent retries for results. Defaults to 1000.
Request.setOutput
Takes a stream to write the results to. No default, if not set results are not written anywhere.
Request.setLogging
Takes a stream to write the logging and errors to. No default, if not set the logs are not written anywhere.