Systems Engineering

Systems engineering is an odd subject and skill which on the surface looks very reasonable but then becomes quite unclear as to what a systems engineer actually does day-to-day. In this guide I’m going to try and explain where you would normally find a systems engineer and what they would do.

What does a systems engineer actually do?

When you pick up a book on systems engineering as you are interested in the idea of a skill that looks after a large complex system. You will read about systems theory and maybe systems thinking where taking different views on a problem leads to different solutions. From here you may search for jobs for systems engineer as you like this idea of systems and you interested in building things properly (engineering) and that’s where it gets a bit complicated as it becomes clear that the role of the systems engineer has three big problems.

Three problems of being a systems engineer

Problem 1 – The actual job is done by others, more defined and common jobs

Imagine you arrive at a job and you are introduced as a systems engineer. The first question you will get will be (and you may be asking yourself this) “what does a systems engineer do?”. If you’ve been reading those books and websites you may reply “I am the interface between the stakeholders and development team” to which the common response is “oh – you’re a project manager”. So you try again “I apply engineering principles to develop and maintain a system” – “oh, you’re an architect or engineer”. Try again. “I define the requirements for a new systems to replace the old one or solve a current problem” – “oh, you’re a business analyst”. And finally “I am responsible for the performance of the system through it’s total lifecycle including the cost to build, maintain and dispose” – “got it – you’re a product manager”.

As you can hopefully see the responsibilty of the systems engineer falls across many more roles that you may have come across. The first challenge for the systems engineer is what balance they will bring as the main role of the systems engineer is to bring the roles stated above into a single view so that when asked “will the solution work?” the system engineer should be able to say “yes – to this tolerances, within these conditions, in this environment”. For the majority of systems engineers their work will fall into business analysis and solution architecture and this brings us on to the next problem.

Problem 2 – you have to be an expert before you can start

The next problem with being a systems engineer is that you will feel you need to be an expert on the system from day one. This isn’t a surprise as you are being brought in as an expert who will be able to check if the system will work. Imagine you get hired as the systems engineer for a e-commerce company say J Curve Commerce. You would be expected at some point to know how well the system is running and how to fix it if it breaks. The problem here is it takes time to get to know both the technical system and, importantly the people that currently run it and the wider business. How much time will depend on the level of experience you already have; if you have worked in e-commerce before then you will know the key components to look out for – website, products sold, inventory, payment, and shipping systems, returns, marketing etc. Therefore as a systems engineer you need to either know the system already or get to know it as fast as possible so you start being useful. In many cases you don’t need to know the specifics of a system but the basic building blocks. Find these and then build them out by asking people how it works (it’s really important to ask people).

Problem 3 – you don’t build anything

Remember I said that the SE role is often done by more specific jobs but you have to be an expert? Well, this leads to being the master of nothing as you know a little bit about everything but not a lot about one specific thing and not even have the stress of money and risk that a project manager has. SE is all about joining things together not building anything which means they have to constantly ask others how things are going. They don’t design anything in detail and build from those instructions as systems engineering is all about the interfaces and the elements which can result in you being the head of testing assuming there isn’t a head of testing. Not building anything can leave the SE without a real sense of purpose and responsibility.

So, again, what does a system engineer do?

The three problems stated above seem like you don’t need or want a systems engineer as the work can be done by other people. It appears impossible to be a systems engineer as you either don’t know enough about the details of the engineering and your likely to be in the business analyst role or you know too much detail and your an architect or a developer.

However, there is a reason you do need a systems engineer. Those jobs that are mentioned don’t overlap very well: the business analyst does the interviews and captures the requirements, these requirements are thought over and a design is made of what could be, the design is developed into a thing and tested, finally the tested thing is moved to production and operationalised. At each step there is documentation and conversation but once a job is done it’s passed over (in agile development the team is supposed to stay together but even here the development and operations team are different even if they are running so called DevOps processes.

Systems Engineer – Product Librarian

This is where the system engineer comes in. They can be involved in the whole process from start to finish and provide a story to the production. There are many ways to do this but the best way is to provide overall responsibility for the documentation that is produced by everyone on the development of the product and provide a summary status on the development options. There are specific things that I’ll cover in a minute but for a starter that is a really good thing to do as it takes the weight of the individual jobs.

System Engineer – Requirements Validation

At the heart of what a systems engineer does is track the system is being developed to the requirements that are defined and agreed by stakeholders and the developers. This doesn’t mean every single requirement as this tends to sit with the business analyst who tracks these but the system engineer tracks the overall functionality of the system in terms of the “ilities”: usability, reliability, securability (ok – that last on is a stretch but you get in the idea). Checking that the system works according to the specifications set is called verification (the truth). These are set and documented to know how the system is supposed to work especially with respect to load.

INCOSE Technical Processes

Requirements and operational functionality are the two of the three parts of the technical processes stated by the International Council Of Systems Engineering (INCOSE). INCOSE state 4 process groups that a systems engineer should be aware of:

  1. Agreement processes
  2. Organisational project-enabling processes
  3. Technical management processes
  4. Technical processes

The first three you really don’t need to worry too much about as they are to do with contracting, setting up an organisation to deliver products and management of product delivery (project management). The one that has the vast majority of systems engineering is the technical processes.

The 14 Technical Systems Engineering Processes

INCOSE state 14 technical processes that help the other jobs that are in play when developing a new system or product. Each process follows an input > process > output (IPO). This helps coordinate

  1. Business or mission analysis process
  2. Stakeholders needs and requirements definition
  3. System requirements definition process
  4. Architecture definition process
  5. Design definition process
  6. System analysis process
  7. Implementation process
  8. Integration process
  9. Verification process
  10. Transition process
  11. Validation process
  12. Operation process
  13. Maintenance process
  14. Disposal process

Now. I know what you’re thinking. This all looks a bit … old and slow in this day and age of cloud computing, apps, and fast paced life. This is true and that is why it’s super important as a systems engineer to know these processes and their inputs and outputs to see what is needed when and where. I’ll go through each process separately so it is a little easier to read and follow. Let’s begin.

The Big SE Picture

The first thing to do is not look at all those processes and think it’s all too much. Remember the size of the SE task depends on the complexity of the system so for many projects these processes can be done quickly or started and iterated or be done by a specialist in a particular process.

We can break the 14 processes into 2 major parts: 1) define requirements and 2) prove functionality. Processes 1 – 6 focus on defining requirements, processes 7 -14 focus on proving functionality including disposal as that is a function of the system.

This split is displayed with the requirements going down until it gets to process 7 and then bouncing back up to process 14. In reality SE will cease once the systems is up and running for a while (process 13). With this in mind we can concentrate on processes 1 – 12) split into the two streams.

Needs to Requirements

Now that we have broken the technical processes into two streams we can further break things down into levels of analysis. Before we do this we need a little definition that is helpful in the long term and that is the difference between needs and requirements.

Needs are specific. Wants are options.

The first thing we need to understand is the need of stakeholders. We say “needs” rather that wants or wishes as we are looking to define hard functionality of a system to solve a problem some times called an opportunity but it’s seen as a problem to solve (like a puzzle) rather than something that is problematic (a pain). With the idea that there is a puzzle problem then there is a “solution” which is the system that will be built, bought, or whatever. To solve the problem we need to know how to solve it and this is why we say the “needs of the system”. The reason we are strict on this is that we only want what system needs to do as a need has an impact like you need air, water, shelter, love (but let’s not get mushy). We don’t say “wants” as a want is something that is additional to a need.

Needs are in the eye of the demander

Now, you may be thinking two things: 1) a need is basically the same as a want: I want to eat some food, I need to eat some food; and 2) a product needs things that are not just needs but wants – I need the phone to make a noise when it’s called, I want too pick my own noise when it’s called i.e. a selectable ring tone.

On the first point they are similar but a need is to fulfil a specific (definable) problem whilst want can be a range to meet that need e.g., I need to eat; I want something hot. On the second point is a good one and needs careful consideration in two scenarios. Scenario 1 is where a need to one person can be a want of another. If I extend the food example, one person can say “I want to eat something sugary like a chocolate bar”, another person can say “I need to eat something sugary like a chocolate bar”. Hopefully you can see the difference as the former is a preference whilst the latter is something with specific properties perhaps to treat a specific biochemical need i.e. diabetes. So needs and wants have to clarified to understand why they are saying what they are user is saying.

I want it now! Meeting customer demands

I want the world. I want the whole world. I want to lock it all up in my pocket. It’s my bar of chocolate.” Veruca Salt, Charlie and the Chocolate Factory Film, 1971 based on the book by Roald Dahl

This brings in the next scenario which is focussed on a product’s features and the demands of a customer. In the quote above you can hopefully work out that Veruca “wants” the bar of chocolate, she is demanding it – her desparate father may think she needs it. In reality Veruca wants the chocolate bar so she has ownership and power over it. This is her need. For a different reference Golum from the book Lord of the Rings states “We wants it. We need it. We must have the precious.” Again the need is to alleviate a greed.

This greed is a fundamental of marketing products – create a greed for a product as a greed changes a want into a need even. When working on designing products we can capture what a customer wants, desires, finds cool, will pay more for, etc to create better products. However designers and engineers can not build a want as we can’t test this want due to the emotional load and focus on the properties the system needs in order to operate to a specific standard such as quality of the product and the production process, legal and financial obligations, health and safety, staffing skills etc. You get the idea. We will get to this later but wants are emotional and can be validated, needs are functions which can be verified.

If I can summary: wishes > wants > needs

Requirements are testable needs

Hopefully by now we have set the scene for how we are going to take the aspirations and thoughts on how something may be can be defined at different levels. The next step is to convert the needs into requirements. Now, before we head down this hill we need to define ‘requirement’.

A quick dictionary search of the word requirements we get definitions like:

requirement – 1) a thing that is needed or wanted, 2) a thing that is compulsory; a necessary condition

In traditional spelling bee tournaments to use it in a sentence: “it is a requirement you have a valid licence to hire a car”, “I require a signature”, “production met the requirements to be certified”. From these definitions a requirement is something that is necessary or condition – if you don’t have it you don’t pass. If we take this definition if a requirement is a test to see if it’s true. When we talk about true or false, yes or no, then it’s something to be verified. Requirements get verified and this is very important in systems engineering as system engineering is only used really for really big systems due to the overhead of doing these processes (you can do it with smaller systems if you’re want). Now, there is another test in system and product development and that is the user accepting the thing built meets the need stated. This is called validation which means a person stands by a property, thing, person – “I validate his character”, “She was validated in her actions” – both of these are subjective views but we’ll get to that. For now when we define requirements we to know how to verify them.

Requirements can be difficult to nail down to test

That’s the theory and it would be nice to say a requirement passes or fails but in reality it can be hard. For first challenge is that you have to ask the person with the need state how to test the need and they may not know, or do know and don’t want to say, or are told how to test it, or there is a range of acceptance. The second challenge is that even if you can state them they way be difficult to actually test. The third and final challenge is there is often pressure on the developers and testers to say that the system is acceptable or not all the requirements accepted are need at the end, which can be true, leading to a sub-standard product in production a painful thing to see from a systems engineering point of view.

These three things may look like a requirement can never be accepted (it can be tested) as the acceptor could argue that is not was meant. Whilst this can happen it is rare as both the stakeholder and developer wants to solve the problem. The key to reducing the likelihood of disagreement are two fold: 1) document what has been agreed (reduces the risk of misunderstanding of what was needed) and 2) test the needs early to confirm the need stated (reduces misinterpretation of a stated need). Both these points mean a disciplined approach is needed which can seem strict but it’s important that development of the solution is defined against the problem stated. How requirements are managed I’ll get to and how the system is developed is covered in a different set of INCOSE processes call Technical Management Processes which work in tandem with technical processes (the 14) and organisational processes such as working with the portfolio.

Levels of Wants, Needs, and Requirements

That’s a lot about needs and requirements but we’re not quite done yet. The last thing we need to do is to express the needs and requirements at different levels of an organisation as these differ depending on the level. Before going further it is really helpful to keep in our minds that levels within an organisation are not fixed as if information has to go through levels to move around. The other thing on the levels is that we often here the phrase “high level view”. This can cause confusion as it’s not always clear what high, level, or view actual means. Often it means – from high up meaning seeing more but with less detail. It’s important to not confuse this with a view from a senior of high level within the organisation as these are not the same thing with the latter being a view from senior or high level.

There are five common levels that INCOSE define:

  1. Enterprise
  2. Business Management
  3. Business Operations
  4. System
  5. System Element

Now, you can get a little excited or frightened by these levels especially if you have a small business.

Enterprise means the people that own the organisation or report to the owners. This level can be called the executive and they are normally independant of other stakeholders. This means they can be an executive of a particular business department or region. However they are defined they are senior members ofter described as the top of the organisation.

Business Management means the people that execute the direction of the strategy. Managers of the management level often make up the executive or the executive committee (ExCo). The managers are generally in charge of a specific business function and are responsible for it’s action and performance. They are likely to be the people that either will use or sell the system that will be built.

Business Operations is similar to management and are often combined in smaller organisations. Business defines a separate business function in a specific area or location. These business units can be legal business units registered as a business within a larger group or can be a specific business unit that operates independently but it’s core operations provided by the wider organisation.

System is the thing that is being built. It can involve often does involve people who use the system or it can be a standard alone system that works by itself.

System Element are the components, bits, parts, etc that make up the system. The system elements can be processes in themselves not not just physical components.

There is a fifth level that should be considered and that is data. Data can be seen as a system element but as it can move and be transformed it can be tricky to nail down so just remember it’s something to consider as we move through the processes.

Each of these levels has corresponding processes to keep the right level of detail at the right level.

  1. Enterprise – how will the system affect our effectiveness against the cost and risk of not doing it?
  2. Business Management – how will we use the system to make the organisation better (more certainty, more profitable, more organisation goal).
  3. Business Operations – how will this system work for this part of the business and with other parts of the business
  4. System – how with the thing works by the organisation – the tool/product/system being built.
  5. System Element – how a component of a system work with respect to inputs, process, outputs and tolerances.

Requirement levels: Enterprise > Management > System > System Element

With these different levels needing different information the technical processes are divided to create the necessary views for the levels we have talked about

Org LevelNeedsRequirements
EnterpriseGrow and secure organisation
ManagementTargets and goals
OperationsMaterials, Skills, and Tools
SystemTools and Processes
System ElementTools and Process components