You understand the potential of Generative AI (GenAI) to unlock new insights and transform business processes. However, experimenting with AI often requires extensive technical expertise, high upfront costs, and...
Shane Gibson Blogs
Because sharing is caring
Automate the Dross
Your analyst's time is valuable, they should be spending their time discovering insights and driving insights into decisions—not manually handling data collection, validation, and processing. With AgileData’s “Automate...
Answer a Burning Question
Making informed decisions quickly is essential to driving your organisation forward. Yet, relying on gut feeling or lacking the data to support decisions can lead to missed opportunities or uncertainty. AgileData’s...
Unified Data Platform
Your legacy data platform may be holding your organisation back. Managing outdated systems, expensive infrastructure, and complex data operations is time-consuming and costly. AgileData’s Unified Data Platform is...
Google Cloud without complexity
Moving to the cloud promises flexibility, scalability, and innovation. But for many organisations, the complexity of managing Google Cloud infrastructure can overshadow these benefits. AgileData simplifies your journey...
NZ Scaleup AgileData achieves Google Cloud Ready – BigQuery Designation
AgileData has achieved Google Cloud Ready – BigQuery designation, streamlining data management for customers and partners. This certification confirms the integration’s functionality and reliability, reducing complexity through a low-code interface. By leveraging Google Cloud’s infrastructure and BigQuery, AgileData empowers business leaders to rapidly gain insights and make informed decisions efficiently.
The last (for now) of our #AgileDataDiscover summaries!
Shane Gibson and Nigel Vining completed a 30-day public experiment using a Large Language Model for legacy data warehouse discovery. They confirmed it was feasible, viable, and valuable, securing their first paying customer. They’re showcasing their progress at Big Data London in September under their new product, AgileData Disco.
#AgileDataDiscover weekly wrap No.5
We are in the final phase of building a new product, AgileData Disco, aimed at efficiently discovering and documenting data platforms. We are exploring various Go-to-Market strategies like SLG and PLG. Pricing strategies include options like pay per output or subscription models. We are building in public to gather feedback and refine their approach.
#AgileDataDiscover weekly wrap No.4
We review feedback, highlight emerging use cases like legacy data understanding, data governance, and automated data migration. New patterns are needed for moving from prototype to MVP. Challenges include managing tokens, logging responses, and secure data handling. The GTM strategy focuses on Partner/Channel Led Growth.
Data Clean Room
Organisation Intent Enable the organisation to share, combine, and analyse data with partner organisations while keeping every byte private and every regulation satisfied, removing the fear and friction that hold back...
#AgileDataDiscover weekly wrap No.3
We focus on developing features such as secure sign-in, file upload, data security, and access to Google’s LLM. Challenges include improving the menu system and separating outputs into distinct screens for clarity. Feedback drives their iterative improvements.
#AgileDataDiscover weekly wrap No.2
We discuss the ongoing development of a new product idea, emphasising feasibility and viability through internal research (“McSpikeys”). Initial tests using LLMs have been promising, but strategic decisions lie ahead regarding its integration. The team grapples with market validation and adjusting their workflow for optimal experimentation.
#AgileDataDiscover weekly wrap No.1
We are tackling challenges in migrating legacy data platforms by automating data discovery and migration to reduce costs significantly. Our approach includes using core data patterns and employing tools like Google Gemini for comparative analysis. The aim is to streamline data handling and enable collaborative governance in organisations. Follow their public build journey for updates.
We are working on something new at AgileData, follow us as we build it in public
The AgileData team is dedicating 30 days to exploring a novel data use case, which might lead to a new product, feature set, or module. They’ll document their daily progress publicly to share learnings and insights. Follow their journey on their blog for updates as they build and experiment in real-time.
Introducing Hai, AgileData 2024 Data Intern
I’m Hai, a name that intriguingly means “hi” in English. Originally from Vietnam, I now find myself in Australia, studying Data Science and embracing an internship at AgileData.io. This journey is not just about academic growth but also about applying my knowledge in practical, impactful ways. Join me as I explore the blend of technology and community, aiming to make a difference through data.
Defining self-service data
Everybody wants self service data, but what do they really mean when they say that.
If we gave them access to a set of highly nested JSON data, and say “help your self”, would that be what they expect?
Or do they expect self service to be able to get information without asking a person to get it for them.
Or are they expecting something in between.
I ask them which of the five simple self service patterns they want to find, which form of self service they are after.
There are 3 strategic / macro data use cases
I often ask which of these three macro data use cases the Organisations believed were its priorities to achieve their business strategy:
Providing data to Customers
Supporting Internal Processes
Providing data to External Organisations
Each of these three strategic / macro data use cases come with specific data architectures, data work and also impact the context of how you would design your agile data ways of working.
Eventually the data maintenance Tortoise will catch the new data work Hare
When you work in a data team you have to split your time between building and delivering new Information Products and maintaining the ones you have already delivered.
DataOps patterns can help reduce the time you spend on the maintenance work.
Building the Data Plane while flying it
In the data domain you typically have to balance between building the right thing and building the thing right.
The days of being able to spend 6 months or a year on “Sprint Zero” creating your data platform have gone.
One team I worked with called it “building the airplane as you fly it”
Here are 5 patterns I have seen data teams adopt to help them do this.
2024 the year of the Intelligent Data Platform
AI was the buzzword for 2023 and it will continue to be the buzzword for 2024.
I have been thinking about our approach to AI in our product for a while and landed on 3 patterns that I use as a reference.
Ask AI
Assisted AI
Automated AI
Adopting these patterns moves a data platform from being a manual data platform, towards a data platform that can do some of the data work for you.
An Intelligent Data Platform.
The 3 patterns of AgileData AI
Having AI embedded in your product have become table stakes it seems.
I have been thinking about our approach to AI in our product for a while and landed on 3 patterns that I use as a reference.
Ask AI
Assisted AI
Automated AI