Littlstar Hivemind Save

For creating distributed jobs using AWS Lambda functions

Project README


For creating distributed jobs using AWS Lambda functions.


npm install lambda-hivemind

Quick Example:

const Hivemind = require('lambda-hivemind')

const hivemind = new Hivemind({
  func: {
    name: 'buzzbuzz',
    handler: 'buzzbuzz.sting',  // '<name of file>.<exported func name>'
    role: '<arn of IAM role to run function under>'
  accessKeyId: '', // AWS credentials. Optional.
  secretAccessKey: '' // AWS credentials. Optional.
  awsRegion: 'us-east-1',
  data: [
    'bee 1',
    'bee 2',
    'bee 3',
  chunkSize: 10  // Size of chunks to split data into

// Oh no, we haven't published a function of this name yet!

// List all files you want published as your function
  files: [
}, {
  // Any AWS parameters specified under

hivemind.on('create', () => {
  // Ok, let's go!

hivemind.on('finish', (results) => {
  // Each job's results will stream to here.

hivemind.on('end', () => {
  // All jobs have finished!

hivemind.on('error', (err) => {
  console.log("Aww, man!")




  • func - Description of function
    • name - Name of the function
    • handler (optional) - Handler of function (If your file is named index and your function is named handler, your handler will be index.handler) (default: index.handler)
    • role - ARN of AWS Role to run function under
    • runtime (optional) - What framework to run function with (default: node6.10)
  • lambda (optional) - Overload of AWS Lambda class
  • data - Array of data to run on
  • chunkSize - How many pieces of data to give to each function
  • accessKeyId (optional) - AWS Access key
  • secretAccessKey (optional) - AWS Secret key
  • awsRegion - AWS region to execute your function in

publish(params, awsParams = {})

Publishes or updates specified code as a Lambda function. This method is not required if the function has been previously published and does not need to updated.


  • zipFile - Relative path to ZIP file to publish as function. This method is required if your script has dependencies. See Code.S3Key below if your ZIP is over 10MB.
  • files - Array of relative paths to files to publish
  • lambdaFunc - JavaScript function to publish as your function. The function must use take 3 parameters (event, context, callback). Data is loaded under event, callback is called when function is done.

AWS Parameters:


Chunks and runs data given in constructor through the function specified in constructor and published in publish()


AWS credentials in constructor are only optional if you have the AWS CLI configured or have your credentials as environment variables!

Full example can be found under test/ folder.

Open Source Agenda is not affiliated with "Littlstar Hivemind" Project. README Source: littlstar/hivemind
Open Issues
Last Commit
6 years ago

Open Source Agenda Badge

Open Source Agenda Rating