Sooner or later every marketing, product or ecommerce team hits the same wall. You need data you don’t own. Competitor prices, search rankings by suburb, job ads, property listings, reviews. The information is sitting on public websites, and the only real question is how you get it into a spreadsheet or a dashboard without losing three months.
There are two ways to do it. You can pay someone who already collects it, which is the data as a service route. Or you can build your own collectors and gather it yourself. Both work. Both go wrong in predictable ways. This article walks through what each one actually involves so you can pick with your eyes open.
What data as a service actually means
The idea is borrowed from software. With SaaS you stopped installing and patching applications and started renting them. With DaaS you stop sourcing and maintaining data and start subscribing to it. A provider collects the raw material, cleans it, removes duplicates, keeps it current, and hands you the finished product through an API, a scheduled file drop, or a share straight into your data warehouse.
If you want the fuller background, ProxyEmpire has a clear explainer on data as a service that covers the delivery models, the types of data sold this way, and what sits behind a web data feed. It is worth ten minutes before you talk to a vendor.
One small warning for anyone searching the acronym: DaaS also stands for desktop as a service, which is about hosted virtual desktops. Same letters, completely unrelated product.
The case for buying a dataset
Buying wins on speed, and it isn’t close. A decent provider can have a sample in your inbox this week. Building a collector for the same data means scoping, writing, testing and babysitting it before a single clean row arrives.
You also hand off the boring part. Websites change their layouts without telling anyone. When that happens to a dataset you bought, it is the vendor’s engineer who gets woken up, not yours. For a small team this matters more than the price tag, because the hidden cost of collecting your own data is rarely the build. It is the upkeep.
Buying makes the most sense when:
- The data is standard. Company records, broad pricing feeds, keyword volumes, the things lots of businesses want in roughly the same shape.
- You need it quickly, or only for a one-off project.
- Nobody on the team wants to own a scraper for the next two years.
- The data supports a decision but isn’t the thing you sell.
The catch is that you are buying what everyone else can buy. If your competitor subscribes to the same feed, the dataset stops being an advantage and becomes table stakes. You also take the vendor’s choices as given: their sources, their refresh schedule, their fields. If you need prices checked hourly and they refresh daily, no amount of asking nicely will change the product.
The case for building your own collectors
Building is about control. You decide which sites, which pages, which fields, which locations and how often. If you want the delivery fee shown to a shopper in Perth rather than the national average, you can go and get exactly that.
That level of control is the reason companies whose product depends on data nearly always collect at least part of it themselves. A price comparison app, a rank tracker, a rental market report: for these businesses the data is the product. Renting it from a third party who also sells it to rivals is a shaky foundation.
Building makes the most sense when:
- You need sources or fields no vendor covers, which is common for niche industries and smaller local markets.
- Freshness is a selling point and you need to set the schedule yourself.
- The data is your competitive edge, so you don’t want it on anyone else’s price list.
- You have, or can hire, someone who will own the pipeline long term.
What building really costs
This is where most build plans come unstuck. Writing a scraper that works on a Tuesday afternoon is the easy bit. A junior developer can do it. The hard part is everything around it.
Access. Send a few thousand requests from one server and most large sites will slow you down or block you outright. Collection at any real volume runs through proxies, usually residential ones, so requests are spread across many ordinary household connections instead of hammering from a single address.
Location. Prices, stock levels and search results change depending on where the visitor appears to be. If you sell into Sydney, Auckland and Singapore, you need to see each of those versions, which means choosing where your requests come from.
Maintenance. Sites redesign. Parsers break quietly. A field that used to hold a price starts holding the word “Sale”. Someone has to notice, and ideally before the client does.
Quality checks. Gaps are the thing that kills trust in a data product. One missing day in a pricing history is worse than a slightly late delivery, because nobody can get that day back.
Storage and delivery. Raw pages, parsed records, backups, and some way of getting the result in front of the people who asked for it.
None of this is exotic. It is simply more work than the first estimate suggests, and it never fully finishes.
Buy vs build at a glance
| Question | Buying a dataset | Building a collector |
| Time to first data | Days | Weeks to months |
| Upfront effort | Low | High |
| Ongoing work | The vendor’s problem | Yours, permanently |
| Control over sources and fields | Limited to the catalogue | Complete |
| Uniqueness | Competitors can buy the same thing | Yours alone |
| Cost pattern | Predictable subscription | Staff time plus infrastructure |
| Main risk | Coverage gaps and licence limits | Breakage and missing days |
Five questions that usually settle it
- Does a dataset already exist that covers what you need? Check before you assume. If one covers 90 per cent of it, buying and living with the gap is often the smarter call.
- Is this data what makes you different? If yes, lean towards building. If it is background input, lean towards buying.
- How fresh does it need to be? Weekly reporting is easy to buy. Near real time monitoring of specific pages usually isn’t.
- Who will own it in a year? A collector without an owner is a liability. Be honest about whether that person exists.
- What does the licence let you do? Some datasets can’t be resold, stored beyond a set period, or used to train models. Read that clause first, not last.
The middle path most teams end up on
In practice the answer is rarely one or the other. Most teams that have been through this a couple of times land on a mix: buy the standard datasets everyone needs, and build collectors only for the handful of sources that set them apart.
An online retailer is a good example. Broad market data and category benchmarks can come from a subscription. The twenty competitors that actually matter, checked every morning for the specific products the retailer stocks, are worth collecting in house, because no off-the-shelf feed will match that list exactly.
The mix also gives you a sensible way to start. Buy first, learn which fields you really use, then build only the parts where the purchased data keeps falling short. That beats spending a quarter on a custom pipeline and discovering you only needed three columns.
If you do buy, test before you sign
Ask for a sample or a trial period and treat it the way you would treat data from a new hire. Pick a few records you know well and compare them with the live source. Count the missing days. Ask where the data comes from and how it was collected, and how you will be told when the format changes.
A good vendor answers those questions quickly and without getting defensive. A vague answer on sourcing is a reason to keep looking.
A quick word on staying on the right side of the rules
Whichever route you take, the same basics apply. Stick to publicly available pages, don’t go around logins or paywalls, keep request rates reasonable, and be very careful with anything that identifies a person, since privacy law applies whether you collected the data or bought it. This is general information and not legal advice, so check your own situation before you launch anything.
Wrapping up
Buying a dataset gets you moving fast and keeps the maintenance off your plate. Building a collector gives you control and something your competitors can’t simply purchase. Neither is the grown-up choice by default.
Work out which data is merely useful and which data is the reason customers pick you. Buy the first kind. Build the second. And revisit the split once a year, because what counts as standard keeps changing.
