For the purpose of this case study, the Shipyard customer has chosen to remain anonymous. They’ll simply be referred to throughout as Twin Benefit. The Shipyard user will be referred to as Tim Kurtz. Should you wish to use them as a reference when evaluating Shipyard, that can be arranged.
Twin Benefit provides a virtual platform for companies to sell its handcrafted merchandise.
The company has grown nearly 400 percent over the past three years, according to Inc. 5000.
Yet when Tim Kurtz, Data Scientist, joined Twin Benefit in 2021 he found himself having to build a data plan, add data tools and systems, and establish foundational processes.
“Twin Benefit was a very mature company in most ways when I joined,” said Tim. “But we weren’t data mature. We had the data, but no way to get it to stakeholders.
“My immediate work was quick and dirty and involved building Python scripts to get data transformation and delivery in place.”
But Tim knew that he had to make the processes cleaner, get observability and logging into place, and automate their workflows. It was too much manual work, and it wasn’t going to go away.
This was when Tim began the search for a data operations platform that was easy to use, could quickly be deployed, and was powerful.
Easy to Use
At the beginning of his search, Tim was referred to Shipyard by his CTO.
“We have limited resources, and what’s most important to us is providing value and not coding or putting out fires. I’m one person, so being easy to use while not sacrificing power was critical.
“The first thing about Shipyard that really jumped out at me was that if you have code that runs well locally, it’ll run on Shipyard,” said Tim. “This was really important as I had Python code on my local machine. Thanks to this unique Shipyard feature I didn’t have to write custom code or make changes. It just worked exactly as I wanted it to work.”
Tim also was impressed with Shipyard’s GitHub integration. “When I do have changes to our code, I push it to GitHub and I know our workflows are always going to run the updated code.”
Shipyard’s drag and drop interface was also a draw for Tim. “Building and modifying pipelines visually in Shipyard is very easy,” said Tim. “It was extremely complicated on many of the other platforms I considered. This was another important area where Shipyard really differentiated itself.
“Choosing Shipyard ended up being a really easy decision. It aligned perfectly with our requirements.”
Quick Deployment
“So many of the tools I looked at took weeks, usually months to deploy,” said Tim. “They were also very complex and required a lot of custom coding. These just weren’t going to work for us.”
“Moving my Python code and deploying Shipyard happened very quickly. It took maybe a day or two,” said Tim. “Shipyard also helped in setting up a Spark environment, which played a crucial role during this process.
“Pipelines were running and Shipyard was integrated with our legacy databases and our primary data warehouse, Snowflake. The ease of adoption allowed for the automation of complex tasks without much manual intervention.”
Today, Twin Benefit executes 22.7 hours worth of workflows each day. Since partnering with Shipyard, they’ve executed nearly 15,000 hours worth of jobs. “It’s been almost three years with Shipyard, and our pipelines have been running smoothly ever since. Today the orchestration of processes is easy and we’ve significantly enhanced the overall efficiency of our data operations,” said Tim.
Twin Benefit’s stakeholders are getting the data they want, whenever they want it, without any interruptions. Whether they’re looking for reports every morning or every hour, Shipyard ensures the data gets delivered on time and accurately. “They now have full confidence in the new pipelines,” said Tim.
Powerful Workflows
“I’ve been able to automate some fairly complex processes in a way that’s really easy for everyone to understand. This has been great for building trust.
“If a stakeholder wants to know what’s going on under the hood, the Shipyard visual UI makes it easy for them to understand exactly what’s happening.
“My work has been totally changed by Shipyard. I'm able to focus more on some of the fun stuff. This means more time to spend on the analytical parts that interest me. Also being able to automate a process and have it triggered on its own is really big.
“And it's kind of needed for some of the processes that I’m now working on. So, for example, we have a clustering algorithm that runs on Shipyard, both the training and the production process. This provides valuable insights for customer segmentation and proactive customer support.
“Being able to let that run freely and make predictions and give our business stakeholders insights. It's kind of the whole reason why I wanted to be a data scientist. So I guess it kind of just speaks to the smoothness of the platform,” said Tim.
Conclusion
Tim has some advice for those considering Shipyard.
“I'd say that it can be a life statement if you're doing manual updates for things that should be automated. Being able to orchestrate pipelines on a platform that's so easy to understand and use is a big benefit.
“I haven't really had to dig into any pipelines yet because something went wrong or some environment didn't really get set up right.
“Everything runs really smoothly and consistently. It's really easy to onboard, very quick. It’s just so easy to use and it has all of the functionality we need and more as we continue growing,” concluded Tim.
