Is There a Return on Investment from Model-Based Systems Engineering (MBSE)? Part 1

Join us for a live webinar on this topic, “Is There a ROI From MBSE” on Thursday, October 17th at 11:00 am ET. Register Here.

This question is one that many people ask. In fact, the International Council on Systems Engineering (INCOSE) has that as one of its tasks for their Value Proposition Initiative. A group of systems engineers is trying to find evidence to prove that MBSE has value. However, that becomes very difficult for a concept that has only been around for a dozen years, when the lifecycle of many of the systems of interest are measured in several decades.

We can approach this question by inference. If there is a significant return on investment in systems engineering, then we can infer that there might be one for MBSE. Fortunately, we do have many decades of experience applying systems engineering to project since the 1950s (at least, depending on how you define systems engineering). One of the best analyses I have come across over the years was a very interesting piece of work by Werner M. Gruhl who was at the time the Chief of the Cost & Economics Analysis Branch at NASA. His work was published in a NASA technical paper entitled, “Issues in NASA Program and Project Management” (NASA SP-6101 (08), in 1994. In a paper by Ivy Hooks in this publication, she states that “if the program requirements are not well understood, there is not much hope for estimating the cost of a program.” She continues: “Werner Gruhl developed a history of NASA programs versus cost overruns” and cited the diagram below (redrawn to due to the poor quality of the document found – an old scanned PDF). She interpreted this chart as “if you have not done a good job in Phase A and B in defining and confining your program, you are going to encounter large numbers of changing requirements and the cost will go up accordingly.” Note the figure below indicates % Spent on Systems Engineering, which it is really Phase A and B in NASA terminology.

Thus, it’s clear that at least the combination of program management and systems engineering, which is what allows you to properly develop the set of requirements for the program, is required to keep the cost of the project from sky rocketing. Note that program management and systems engineering are flip sides of the same coin. The program manager optimizes cost, schedule, and performance, while mitigating risk in each of these areas. The systems engineer is tasked to do the same for the system. That’s why these two disciplines have been seen to overlap, which was recognized in a recent book by INCOSE and the Program Management Institute (PMI).

Another more recent attempt at qualifying the value of systems engineering came from the Software Engineering Institute. They conducted several studies to determine the value of systems engineering, including one documented in their November 2012 paper entitled “The Business Case for Systems Engineering Study: Results of the Systems Engineering Effectiveness Survey.” The authors “found clear and significant relationships between the application of SE best practices to projects and the performance of those project” as seen in the figure below.

Project Performance vs. Total SE Capability

In a later presentation by Mr. Joseph P. Elm, one of the authors of the 2012 paper, on “Quantifying the Effectiveness of Systems Engineering,” he cites a finding for a General Accountability Office (GAO) report (GAO-09-362T) that states:

“… managers rely heavily on assumptions about system requirements, technology, and design maturity, which are consistently too optimistic. These gaps are largely the result of a lack of a disciplined systems engineering analysis prior to beginning system development …”

So, it is recognized that there is great value in performing at least the “right amount” of systems engineering. If we use the Gruhl graph as a basis, we need to spend around 7-12% of the program’s budget on the combination of program management and systems engineering. Since the cost of the program could as much as double on average according to the chart, the return would be about 10 times the investment. For example, if we spent $100,000 on systems engineering and program management and the overall cost of the program was $1,000,000, then since the cost could have doubled to $2,000,000, we save $1,000,000.

So now that we agree there is a substantial return on investment in systems engineering, let’s get back to the question of ROI on MBSE. The question here is then, “does modeling help systems engineering.” Since we have always done modeling in systems engineering, I think that is clearly a part of good systems engineering. But the flavor of MBSE being pushed by many in the community has equated MBSE to SysML and many have also equated SysML as implemented by MagicDraw. But does SysML and MagicDraw® do all the things we need to do in systems engineering? In particular, do we obtain a good set of system requirements in a form easily used by all the stakeholders?

To begin to answer these questions, let’s go back to Mr. Elm’s paper. He states that the systems engineer must perform the following tasks:

  • Requirements Development
  • Requirements Management
  • Trade Studies
  • System Architecture Development
  • Interface Management
  • Configuration Management
  • Program Planning
  • Program Monitoring and Control
  • Risk Management
  • Product Integration Planning and Oversight
  • Verification Planning and Oversight
  • Validation Planning and Oversight

 

SysML consists of nine diagram type, most of which were derived from software engineering practices, not systems engineering. Yes, there is overlap between the two, but not as much as the overlap between systems engineering and program management. That becomes obvious from the task list above, many of which include explicitly the word “management.”

SysML also has proven to be very difficult for most other disciplines to understand, since they speak other languages. It also takes at least two large books, “A Practical Guide to SysML” and “SysML Distilled.” In comparison, the Lifecycle Modeling Language provides a whole systems engineering ontology and limited diagramming that completely subsume SysML. However, LML can still be explained in a very thin book, “Essential LML.” You can see the comparison in the picture below.

You probably are asking, “How is that possible that LML subsumes SysML?” By using an ontological approach that defines a set of entity classes and their relationships, along with the attributes on both, LML provides all the elements of a real language (nouns, verbs, adjectives and adverbs). This ontology can be used to capture the information easily and efficiently. Then that information can be displayed in many ways, including all the nine SysML diagram types.

The Innoslate® tool proves this assertion, as it produces all nine SysML diagrams (and many more) from this ontology as extended in Version 1.1 of the LML specification. In addition to the SysML diagrams, Innoslate produces the LML Action Diagram, that represents the same information as the SysML Activity Diagram, but in a significantly more understandable form. We can see this when we compare side by side the two type of diagrams, as shown below.

LML Action Diagram Example

SysML Activity Diagram

In the SysML diagram I need to know what the diamond and fork symbols mean. In the Action Diagram, I know exactly what they mean, because the words: OR, LOOP, and Decomposed, make their intent clear. In addition, in SysML I cannot just allocate the decision points to who or what performs them. I can in LML. Of course, if there were only two symbols I needed to decipher, then I would not care as much, but SysML has over 30 such symbols. You will need a “3-D decoder ring” to fully understand how to use all the symbols, hence the very large books and long training classes needed to try to learn SysML. This learning curve translates into a significant investment in the workforce to get them up to speed on this complex language. Of course, the electrical engineers, mechanical engineers, logistics experts, and all the other disciplines have their own languages and have no interest in learning something this complex.

You might say, “but of course MagicDraw overcomes these limitations in SysML?” The answer to that is not as well. In particular, if we go back to the list of systems engineering tasks, MagicDraw only does one “well:” System Architecture Development. Although MagicDraw has some limited requirements capability, almost everyone uses another requirements tool in conjunction with it. Innoslate by comparison has a robust Requirements View that includes automated requirements quality checking using the artificial intelligence technique of Natural Language Parsing (NLP). The requirements then can be directly traced to the diagram entities within the same tool, resulting in one database and no configuration management problems that you encounter having two databases. Innoslate also has a built in Test Center for the V&V activities. In addition, Innoslate provides Discrete Event and Monte Carlo simulators to verify that the Action (or Activity) Diagrams have been correctly done. We use that same approach to support Program Planning, Monitoring and Control.

So back to the ROI discussion. Can MBSE provide a healthy ROI? Only if we do all the things we need to do in systems engineering. In addition, if we can use modern technology to help automate these difficult tasks, we can provide even higher ROI than systems engineering by itself. Innoslate and LML provide a means to provide this higher ROI, while MagicDraw and SysML actually cost much, much more to implement and you end up with a poorer result. So, if you want ROI from your MBSE investment, use Innoslate and LML.

Join us for a live webinar on this topic, “Is There a ROI From MBSE” on Thursday, October 17th at 11:00 am ET. Register Here.

innoslate, systems engineering, model-based systems engineering, requirements management

Innoslate 101: A Webinar for New Users

Come join us June 6th at 11:00 am EDT for “Innoslate 101: A Webinar for New Users.” Newbie Innoslate user, Joannah Moore, is going to show you just how easy it is to learn Innoslate. She will walk you through the ins and outs of the tool and show you how you can become an expert Innoslate user in no time.

Stay after the live demonstration for a question and answer session with systems engineering expert, Dr. Steven Dam.

Register Today

About Your Host

Joannah Moore is in both Sales & Support at SPEC Innovations. Before SPEC, Joannah’s career was on a strict business path including Commercial Insurance and Property Management. However, years into property management, she was hungry for more. This brings us to 2018 when she joined SPEC Innovations as a recent college graduate with her B.S. Degree in Business-IT Management. She is a certified IT professional with many IT certifications in the various fields of IT, including project management

About Innoslate

Innoslate is the model-based systems engineering solution of the future. An all-in-one software package made for systems engineers and program managers, you can keep your requirements management, modeling and simulation, test management, and more all in one place. Smarter, more successful systems start here. Create a trial account at innoslate.com/signup.

Implement a Strategy for Transforming from Office Products to Model-Based Systems Engineering (MBSE)

Often people will ask me: “Who is your major competitor?” My usual response is “Microsoft Office.” I don’t say this because MS Office is a bad tool … it is not. It is a very good tool for publishing information. I use it often and have become quite the expert in using it to write books, papers, accounting, and presentations. But for systems engineering, it’s not the right toolset. Unfortunately, most people trying to perform SE tasks use MS Office because that’s the only tool they have. It’s cheap and already approved for use by management, but it does not provide all the capabilities SEs need. You can’t easily perform the kinds of analyses we need, such as functional analysis, simulation, requirements analysis, and risk analysis. That’s not to say that you can’t perform these kinds of analyses in the same way as your parents (or grandparents). They did it, but usually it was with armies of people. They did it with relatively simpler systems. They did it using libraries with librarians to help them find the things they needed. If you think I am exaggerating, I actually saw this in effect as late as 1986, before we had extensive implementation of personal computers.

But with the widespread availability of networked, high performance computer systems providing ready access to amazing processing capabilities, and the breadth of the worldwide web, we also have new tools that can do much more. And it’s a good thing, because at the same time this technology has caused system complexity to grow exponentially. We no longer have the “armies of people” and “librarians” available to help us do the work. So we have to do more with less.

So, let’s say your management has finally realized that you need better tools, or they just want to be “fully buzzword compliant” by jumping on the MBSE train. Now how are you going to come up to speed on a new toolset, while still continuing to meet cost and schedule?

The purpose of this paper is to help you make the shift from products like MS Office to a true MBSE tool: Innoslate®. Some of these strategies may be useful in migrating from Office to other tools, but other tools really don’t have all the features you need and when you put together the set you will have to spend a lot more on those other tools. Money being spent isn’t just the cost of the tools, but the people costs of operating them. Much like Office, even the toolsets being offered by others are really just a set of individual tools that were loosely “integrated” to provide a package. That’s why they have so many “plug-ins.”

So on with the strategies.

Strategy 1: Start Slow

Just like you can’t eat an elephant all at once (and I don’t recommend eating elephants at all!), you should migrate your information a piece at a time. For example, perhaps you have a big library of Visio diagrams and you want to reuse in Innoslate®. You might say, “well does Innoslate® have a way to import diagrams from Visio?” The answer is no. The reason we don’t is that tools like Visio are just drawing tools that don’t provide “semantic” meaning. One definition of this term is: “of, relating to, or arising from the different meanings of words or other symbols.” In other words, drawings require a significant set of rules to enable them to represent the information. An example of this from flowcharting is the use of a diamond to represent a decision or a rectangle to represent a process. Both the writer and reader of these diagrams must fully understand those representations for the chart to have any meaning. Unfortunately, with a pure drawing tool like Visio, people will use these symbols incorrectly or in different ways, which makes it difficult for the reader to really know what the writer meant. We have the same problem with unstructured words as well. Writers will use obscure words or use the words incorrectly, which interferes with the communication.

This problem is why MBSE has taken off as a concept to enhance the communications. The tools can help enforce the rules for diagrams. The diagrams can even be analyzed automatically by the computer algorithms to suggest improvements to the diagrams to make them more compliant to the rules.

So, what does this mean? It means that the diagrams as drawn in Visio are likely in error. Just moving the boxes and lines over to another tool will also bring all the errors with them. What should you do?

We recommend taking a few of the diagrams you are using right now, put them on one side of your desk (or in your second monitor, if you have one), and start a new diagram in Innoslate®. We recommend starting with the Action Diagram for a flow or process chart. You will need to interpret the information that’s on the current diagram to create the new one. You will want to take advantage of the capability in Innoslate® to decompose Actions, thus simplifying large diagrams. That allows you to identify subprocesses, which may be repeated in various parts of the diagram. In this way, you will gain a better understanding of the tool and the limitations (rules) that govern the diagrams.

 

Strategy 2: Only Start on a New Task

In this approach you keep legacy information separate or use the Innoslate® Artifact class entities to store the files from previous work, so you can find them if you need them. If you don’t have a strong requirements document from your customer (and you usually don’t), we recommend again starting with the Action Diagram and capture the current operational or business processes using that approach. The purpose of these models is to identify where the problem areas exist and then you can postulate solutions to those problems.

 

Strategy 3: Start with an Innoslate® Workshop

An Innoslate® Workshop provides a means to make learning Innoslate® easier by having our trainers work directly with your problem as the basis for the training. The training is tailored to your processes, situation (such as the phase of development), and problem so that enables the training to have greater relevance to your people. It has the added benefit of helping you get started with solving your particular problem. Don’t worry about us knowing too much about your business. We are happy to sign any appropriate non-disclosure agreements (NDAs) and our personnel have the necessary clearances to help you with your problem at any level of classification. SPEC Innovations is a woman-owned small business, so the chances of a conflict of interest is negligible.

 

Start Any Time

This in a sense is not a strategy, because it applies regardless of the situation. Timing is never perfect for moving from one way of doing things to another. The sooner you get started the sooner you can reap the benefits of MBSE. Just get in there and apply any or all of the strategies above to get started. Let us help you today. We will help you make it as painless as possible. Ben Franklin said: “There are no gains without pains.” It applies here as well. Just like starting your exercise program in the new year, the sooner you start, the sooner you feel better.

Innoslate Cloud vs. Innoslate Enterprise

This article is meant to help you determine which of these solutions is best for you and your team.

 

Innoslate is offered as a cloud or on premise solution, Innoslate Cloud and Innoslate Enterprise, respectively. It can be difficult to decide which one is right for you. We’ll take you through the pros and cons of each one to help you make this decision.

 

Innoslate Cloud

Innoslate Cloud is flexible and affordable. You can get started quickly, since there’s no download required. It’s easy to share projects with reviewers that are not Innoslate users yet. The pricing plans can be monthly or yearly, so you only have to pay for what you need. Innoslate Cloud is hosted in data centers audited for ISO 27001 and SAS70, so security isn’t a problem.

 

Pros:

Scalability through the cloud

High availability

Flexible pricing plans

Affordable pricing

No download required

 

Cons:

No floating licenses offered

Less administration controls

Less options

 

Innoslate Enterprise

Innoslate Enterprise offers scalability and collaboration beyond the cloud and behind your firewall, with the heightened security of your own server. Innoslate Enterprise is available on unclassified (U/FOUO) and classified (SECRET) level government networks through NSERC.  Innoslate Enterprise is also available on Amazon AWS Marketplace. You can choose between floating and named licensing options. You can get started within seconds on a modern server hardware. The high performance and close proximity of Innoslate Enterprise removes latencies. The server, RedHat Wildfly, implements the latest enterprise JAVA standards with full JAVA EE7 support. Wildfly runs on Windows, Mac, and Linux servers.

 

Pros:

Massive scalability (10 million entities)

Increased administration control

More availability through NSERC and AWS

Floating licensing options available

Fewer latencies

 

Cons:

Less flexibility through pricing

More expensive than Innoslate Cloud

Download required

 

Innoslate Enterprise is perfect if you need your SaaS on your organization’s server or want more administration controls through LDAP. Some organization will have to choose Innoslate Enterprise due to strict security requirements. Check with your IT department to see if you are required to use an on-premise solutions.

 

Still not sure, which solution is right for you? Give us a call at 571.485.7800.

Why We Built Innoslate – About Us

As systems engineers who had been using modeling and requirements tools for decades, we kept running into the same problem: we needed a solution that spans the entire system lifecycle. Effective requirements analysis and management requires not only capturing and managing the requirements provided by the customer, but also the analysis and decomposition of those requirements into specifications for the buying and building of the system components.

Requirement Analysis includes modeling the processes and procedures related to the requirements and simulating those processes with realistic constraints in resources, bandwidth, latencies, and many other factors. And once we have a baselined set of requirements, we have to not only “maintain” them (as they constantly change over the lifecycle), but we also have to verify that the resulting components and systems meet the requirements at every level of composition (component, subsystem, system, system of systems, etc.).

We realized what most systems engineers needed was a requirements tool, a modeling tool, a simulation tool, a verification tool, a risk tool, a program management tool, and a document management tool. None of the existing software solutions had native integration for all these different tools. The International Council on Systems Engineering (INCOSE) provides a tools interoperability working group, which includes representatives from all the major tool vendors. Unfortunately, without native integration, users experience missing data points and a drain on time and resources to integrate. It also became unreasonably expensive to purchase so many different tools and to manually integrate and manage each one.  

SPEC Innovations built Innoslate to be the all-in-one solution for systems engineer and program managers. We also wanted engineers to start analyzing overall quality of the entire project early in the lifecycle. That’s why Innoslate provides analytical capabilities like Innoslate’s Requirements Quality Checker and Intelligence View. These two internal features identify problems with the requirements and the overall model, quickly, so they can be fixed early in the process, thus saving time and money. Not only that, you get the scalability and collaboration features you need to work not only small projects, but also very large ones.

How to Make Requirements Approvals Using Innoslate

One question we often get is: does Innoslate do workflow? What people mostly mean by this question is how can they approve documents and requirements and be notified that that approval has occurred? How can we be sure the right person approved the item? Innoslate provides this capability using what we call “lightweight workflow.”

Lightweight work flow follows a simple process using the notifications, labels, history, and baselining features of Innoslate. The steps of that process are provided below.

Step 1: Setting up a signature block.

Most requirements approval occur at a document level. An example would be a standard operating procedure (SOP) for a project. The set of requirements are embedded in the SOP document. Innoslate provides templates for SOPs that can be used to develop them. You only need to access the SOP document (Menu/SOPs as seen below).

 

You can then select a blank document or the Standard Operating Procedure (With Guidelines) and the template will be produced.

In this template, there is a signature block. These can easily be separated in to separate blocks or you can break them out individually.

The screen shot below shows an example of breaking out the signature blocks. We have already added the information about the signatories of the SOP (Project Manager, Program Manager, and Division Manager).

Each manager may want to subscribe to one or more of these block for notification when it has been changed. Subscribing to an entity is easy and available from Entity View. Just select the entity and open the Entity View, as shown below.

In the Entity View of the signature of interest, select the “More” button and “Follow” the entity. The “Follow” tells Innoslate that you want to receive a notification anytime this entity is changed. When a change is made an e-mail will be sent to your e-mail address.

So, once the Project Manager approves of this entity, by adding the “Approved” label (note you may have to add this label as its not part of the standard label set), and the initials or a picture of the person’s signature can also be added at the same time, then an e-mail will be sent out to anyone subscribed to the entity. The e-mail below shows the notification I received from Innoslate.

Going back to the SOP and selecting the Project Manager entity, we see that the approved was made (see below).

The time and date of this approval and who approved it is recorded in the history of that entity, which is accessed from the Entity View using the “History” button (see below).

If you want to ensure that no changes occur elsewhere in the document after each approval, you can use the baselining feature (see the “Baseline” button at the top of the Document’s View).

This action will freeze the changes and only allow changes to the next version.

So using these techniques, Innoslate provides all you need to ensure that the requirements documents you produce follow your workflow.