Should you be building that?

When Shadow IT has some value

Get all the data you need from the database — all of it — and then some more

They kept popping up like whack a mole. These Microsoft Access databases were everywhere. My boss at the time would get numerous calls to fix broken queries and connections to databases (quite often these were access to SQL Server via ODBC). Every time one was fixed and logged, a new one would be found.

Someone’s hobby project was creating a massive support burden for the development team. The culprit was known, they were asked to stop creating these “databi”, but they just couldn’t help themselves. Access monstrosities would be hidden away in secret shared folders that only certain teams had access to.

Enough WAS ENOUGH. How could we get on with the serious work of software development, while dealing with these non-developer tools that had fallen into the wrong hands?

Because it was so quick to create these troglodytes, our backlog bulged. We wanted to productionise these “systems” by binning them all and re-creating them using real programming tools such as ASP.NET. Real tools, for building real systems.

Oh... our arrogance was in overdrive.

From a business perspective the development team doesn’t always know best.

Yes, these systems were horrific, they did things like:

“Get all the data you need from the database – all of it – and then some more. Then you can lop off the bits you don’t need in the client”

But more importantly, they did things like:

“Serve a business need”

Shadow IT

Nowadays there’s a term for this – Shadow IT. The standard definition goes along the lines of:

“In big organizations, shadow IT refers to information technology (IT) systems deployed by departments other than the central IT department, to work around[1] the perceived or actual shortcomings of the central information systems.[2] Shadow IT often introduces security and compliance concerns.”

https://en.wikipedia.org/wiki/Shadow_IT

Now this particular type of shadow IT was technically built by someone within IT. Not a developer, but they did come from the IT department.

But developers can also engage in Shadow IT. I’ve seen many quickly scrape something together to help someone out. A quick patch here and an informal requirement is met. “We’ll just connect that to the internal API instead of the external, that has more functionality.”

This is where Shadow IT gets interesting. At what point does it become just regular IT?

“We can power this API by an excel spreadsheet for speed. We can sort out the database later”

Here the technical solution is morphed so that it can delivered sooner, to meet the business need. Technical debt has increased for the sake of delivery. In a lot of organisations the “later” to sort out the database will be never. Sellotape and plasters everywhere. Eventually making the simplest of changes difficult to implement.

Balance

Fine for a start-up, but for a mature company?

My take as per most things is that balance is needed.

A start-up needs to iterate quickly and speed is key. So does a mature company as well. They need to stay relevant otherwise they will ultimately be consumed. Innovation isn’t born out of IT Surgeries.

But both also need to have a handle on technical debt. As a start-up becomes a mature company, you need to avoid the need to have a total rewrite of your software, that will waste egregious amounts of time and people.

As Elon Musk says “the machine that builds the machine” is the hard thing. That’s what people fail to see when they’re in the detail.

Digital Agencies have a set of tools and templates* so that websites can be quickly spun up without too much effort. Email providers aren’t written from scratch every time a new client comes on board.

* The agencies that don’t just copy and paste everything from the last project ;)

So tools are good. One needs a tool to speed up these things. So that business requirements can be realised without increasing technical debt.

Something like an Access Database would work..

Thank you for reading this article! Please leave a comment below if you have any questions or feedback.

If you enjoyed this article then please consider buying me a coffee:

Chris Sheldon is a Project Manager for DataArt Ltd. DataArt are a global software engineering firm that takes a uniquely human approach to solving problems.

In his career Chris Sheldon has been a Software Developer, Scrum Master, Development Manager and more. He’s decided that people are harder than process, so this is where his attention is now targeted!

Chris graduated in the UK from Reading University with a degree in Electronic Engineering and Cybernetics.

A bit of a Agile enthusiast, productivity nerd and Wantrepreneur Chris really needs to decide what he wants to do in life and focus.

You can connect with him on LinkedIn, Twitter, Facebook, or by visiting his website, ITsChrisSheldon