It's 2021 and the language of data is definitively Python. Just take a look at recent studies from Kaggle, Stack Overflow, and KDNuggets. That doesn't mean it's the best for everything - but it has definitely won the popularity contest.

So what becomes of teams that aren't using Python? When they inevitably have to build out workflows for their data operations, they're getting the short end of the stick. Tools like Airflow and Prefect have established themselves as Python "Workflow as Code" tools, forcing teams to write all of their scripts in Python.

At the end of the day, your coding language of choice is just a preference. And for some teams - that preference is Node.js for scale and flexibility.

At Shipyard, our goal is to make sure that you can launch, monitor, and share solutions that your team is building for data operations regardless of the language that it's written in.

Making Node.js Automation Easier

Today, we're excited to announce that Node.js is now natively supported by Shipyard.

Just like our Python and Bash Blueprints beforehand, Node.js Blueprints enable your data team to:

• Automate Node.js code in isolation without making any updates to your script
• Trigger Node.js code on demand, with a schedule, or with a webhook
• Automatically install packages at runtime from package.json files, or manage package installation through the UI
• Securely pass environment variables to scripts
• Build Blueprints to share and re-use Node.js scripts in a no-code fashion
• Build Fleets that connect your Node.js scripts to other scripts written in any language

Enabling New Data Efforts

The inclusion of Node.js means that your data solutions can take any form and use packages that may make it easier for your engineers to work with Data.

• Use Dataform to classify and clean your data sets in BigQuery
• Use Data Forge to transform your data with ease
• Build CLI tools that can be transformed into Shipyard Blueprints
• Write quick scripts to extract data from APIs

Building Language Agnostic Workflows

Shipyard has been designed from the ground up to be Language Agnostic. That means that you can build scripts in any language and connect them together without running into issues.

Because Vessels within a Fleet share the same files, you can string together larger solutions without having to write each component with the same language or even the same packages. All you need to do is ensure that every Vessel, regardless of it's language, outputs a file that can be picked up by downstream Vessels.

You can even build 80% of a solution with Shipyard's no-code Blueprint Library and then fill in the gaps by scripting in whatever language you choose! Take the following example, where you could build an exhaustive website scraper that loads results into Snowflake and alerts the team via Email.

The possibilities of how you can move and manipulate data are truly endless and with the addition of Node.js, those solutions can be faster and more powerful. As always, even if the language is not natively supported by Shipyard, you can always install it at runtime and execute it using Bash.

Node.js is now available to all subscribers and can be tested with any account. Sign up for our free Developer Plan to start launching scripts that you've written with Node.js. For more information about automating your code with Vessels, view our documentation.

About Shipyard:
Shipyard is a modern data orchestration platform for data engineers to easily connect tools, automate workflows, and build a solid data infrastructure from day one.

Shipyard offers low-code templates that are configured using a visual interface, replacing the need to write code to build data workflows while enabling data engineers to get their work into production faster. If a solution can’t be built with existing templates, engineers can always automate scripts in the language of their choice to bring any internal or external process into their workflows.

The Shipyard team has built data products for some of the largest brands in business and deeply understands the problems that come with scale. Observability and alerting are built into the Shipyard platform, ensuring that breakages are identified before being discovered downstream by business teams.

With a high level of concurrency and end-to-end encryption, Shipyard enables data teams to accomplish more without relying on other teams or worrying about infrastructure challenges, while also ensuring that business teams trust the data made available to them.

For more information, visit www.shipyardapp.com or get started for free.