AFuzion Releases DO-254 Training for Military Avionics Hardware

November 15, 2016 - "Defense organizations throughout the world are adopting DO-254 for their avionics hardware. Why, and what are the implications? Details herein. For private onsite DO-254 training especially for military avionics, see details here (over 9,500 trained in DO-254 and DO-178C, more than all other trainers in the world, combined): http://afuzion.com/avionics-training/workshops/avionics-hardware-intermediate-do-254-training-class/

Folks, this is an increasingly important topic as our AFuzion military clients are adopting DO-254 at an unprecedented rate. We previously only provided this info to our DO-254 training clients, but due to increasing importance we're releasing it here to the general public. Added info is here: AFuzion's DO-254 Training Class Details

For decades, military organizations have developed hardware and software using a variety of specialized, defense-oriented standards including 2167A, 498, and 882. As Military organizations, they were highly motivated to use hardware and software standards which differed from the commercial sector since it was perceived that military applications were "different." Militaries utmost concern was primarily "Mission". Today however, there is an accelerating momentum toward Military/Commercial avionics convergence: adopting DO-178 and DO-254 worldwide. Today, fighter jets (Joint Strike Fighter, T-50, etc), cargo planes (C-130, C-17, A400M, etc.) and UAV/UAS's (formally called RPAS: Remotely Piloted Aircraft Systems) are requiring compliance to DO-178 and new programs increasingly, to DO-254.

What are DO-178 & DO-254? DO-178C is the fourth iteration of the FAA's avionics software standard, required for all commercial airborne software, which contributes to safety of flight by ensuring with a sufficient level of confidence that the software performs its intended functions that have been assigned by the system requirements. For twenty years, commercial avionics software has required certification via DO-178, then DO-178A, and now for over a decade, DO-178. But several years ago, certification authorities realized that avionics safety was dictated by both software and hardware; hardware was just as important as software, but only required adherence to DO-160, the environmental testing standard. So SC-180, the precursor to DO-254, was initiated, thereby levying consistent certification requirements upon hardware. The basis for DO-254 was DO-178 itself, ensuring similarity between certification of software and hardware in terms of processes and objectives to be satisfied.

DO-178 (software) and DO-254 (hardware) presume that software and hardware must operate in harmonic unison, each with proven reliability. Previously, hardware was considered "visible" and tested at the system level with integrated software; hence hardware was exempt from DO-178 quality attributes. But that exemption resulted in functionality being moved from software to hardware for the purpose of avoiding hardware certification. Additionally, hardware complexity has evolved such that hardware is often as complex, or more so, than software due to the embedded logic within the PLDs, ASICs and FPGAs. Now, everyone recognizes that hardware and software comprise an inextricable chain with the quality equal to that of the weakest link, thus the mandate to also apply DO-254 to avionics hardware.

DO-178 and DO-254 utilize five different levels of criticality, ranging from Level A (most critical) to Level E (least critical). Each avionics system is assigned one or more levels of criticality based upon a system safety assessment which analyzes each system's potential contribution to aircraft safety; each hardware and software component within that system must meet or exceed its assigned criticality level. As the criticality level increases, so does the degree of rigor associated with documentation, design, reviews, implementation, and verification.

Previously, military organizations throughout the world utilized their own standards for hardware development. Their rationale for such is listed below. Military supplier management would often time require a CMMI Level 4 or 5 ranking, however commercial certification authorities never gave "credit" for such. Now, today, there is a"Overarching Properties" movement afoot by the FAA (EASA is not fully onboard) which will enable proven companies with a track record and good CMMI rankings (Level 4 is most bantied about) to achieve "credit" - thus these DO-178C and DO-254 Overarching Properties will enable credit for lessened FAA oversight perhaps. Additional reasons for DO-254 compliance include:

Military projects were more complex than commercial projects.

Mission completion is always a highly desirable goal.

Military projects needed higher quality than civilian projects.

Military projects had numerous suppliers to manage.

Military projects required complex integration cycles.

Military projects had long airframe lifetimes to account for.

Granted, prior to DO-178 in the 70's and 80's, the above rationale was valid. However, today, consider the commonality between Military and Commercial avionics:

Both utilize high complexity and complex integrations.

Both utilize hundreds of suppliers with long project lifetimes.

Both require access to leading-edge commercial technologies.

Both are increasingly concerned with re-usability, quality, and
increased cost-effectiveness.

Both require a high level of operability, reliability, maintainability, and safety.

Military aircraft are now utilized more and more in commercial
airspace (they do not want to be restricted in flight paths or hours).

By the year 2003, U.S. military organizations realized that the commercial aerospace sector, particularly those regulated by the FAA via DO-178 (later DO-254), maintained certain advantages, advantages not inherent in the defense establishment. They were faced with a choice:

Maintain the status quo and do nothing.

Update their own Mil standards to adopt the best aspects of DO-178.

Simply adopt DO-178 (and later, DO-254).

What choice was made? Option #3 above. Was it a simple choice? No: as with any established organization, there were myriad opinions, entrenched practices and opposition, added initial transition costs, and politics. The result? A gradual adoption of DO-178 and DO-254 (though DO-254 adoption in the Defense sector lags that of DO-178). Further complicating the military adoption of DO-178 were the following aspects:

The military did not want to relinquish oversight to the Federal Aviation Administration (FAA), nor did the FAA have the bandwidth or authorization to intervene within Military projects.

Military organizations were unfamiliar with DO-178 and DO-254 specifics and therefore applied widely varying and subjective criteria; truth be told, DO-178 and DO-254 are terse and vague­–specialized training by experts is typically required to apply them in the real world.

Because of the above, militaries often further complicate DO-178/DO-254 adoption by requiring simultaneous adherence to their own Mil standards in addition to DO-178/DO-254! While well-intentioned, this action is counter-productive since the standards differ and conflict with each other in key areas; DO-178 and DO-254 already have ample ambiguity and subjectivity which is grossly complicated when requiring corollary adherence to Mil standards.

Unlike Military standards, DO-178 and DO-254 utilize five different criticality levels. Why? Cost. Purely cost. If cost were no object, all avionics software would be designated Level A, the most critical level with the strictest requirements. However, each of the dozens of avionics systems onboard aircraft does not affect aircraft safety to the same degree. The criticality level is chosen via analytical processes which assess the contribution to aircraft safety of each system, sub-system, and component; this criticality level is also based on a combination of engineering judgment, flight experience, and system service life. These safety analyses are covered by their own standards including ARP-4754A and ARP-4761/A for DO-178/DO-254, and are well-known throughout civil aviation. However, such safety analyses are relatively new to Military avionics, hence the criticality level selection subjectivity within defense projects.

Regarding cost, the following graph accurately depicts the cost-delta associated with the different criticality levels:

Costs versus schedule by criticality level are direct consequences of the number and complexity of objectives to be satisfied, however other factors are relevant; it is also a matter of what and how the applicant (industry or organization) perceives and understands the applicable "DO" guidelines as such often yields misunderstanding, mistakes, re-planning and re-work. This is a major cause of the cost increases for aviation compliance.

A popular myth is that DO-254 is expensive. However, Level D certified hardware still has full planning, requirements, implementation, reviews, and basic testing processes applied. Plus configuration management, quality assurance, and DER liaison are applied to Level D. But the costs of Level D should hardly more than any non-certified commercial hardware process. Why? Because Level D is comprised almost entirely of normal industry-standard hardware engineering principals: requirements, tests, and proof thereof.

Another myth is that the most significant cost escalation occurs when moving from DO-254 Level B criticality to Level A. Untrue.

The cost impact of DO-254 is most significant between Level C and Level B. Why? Level B requires the following which Level C does not and which results in Level B requiring at least 50% more budget and schedule than Level C:

Detailed Design documentation (Conceptual and Detailed)

Ensuring 100% coverage of all statements and decision-conditions (or Element Analysis, e.g. of RTL)

Greater rigor placed upon reviews

In many cases more rigorous configuration management

Far more complete traceability is required via 178/254 than traditional Mil standards; most projects use a tool.

Level B requires additional structural coverage (decision-condition, i.e.. all branches in the source code), additional independence in reviews, and tighter configuration management. On first glance then, it would seem that Level A should be significantly more expensive, roughly 50-70%, than Level B. And in theory, such might seem to make sense. But as in many areas of life, common sense overcomes theory.

Level A is the most critical software level and hence the most expensive. True. But another myth exists for Level A, namely "Level A is extremely difficult to achieve and will cost at least 30-50% more than Level B." False or development of hardware; however Level A will require redundancy which means at least twice as much hardware. With proper application of modern structural coverage tools, personnel training, and thorough requirements based testing, the added cost for Level A and B can be contained.

The aforementioned cost deltas are actual, achievable results, as documented by this author on dozens of successful projects as well as leading aerospace providers. However, these cost results are NOT the industry average: the average DO-254 avionics project exceeds these cost deltas by 20-50%. Why? Because of inefficiency, misunderstanding of DO-254, and not applying "Best Practices" to contain costs. This results in re-work and over-work.

Compliance versus Certification

Since the FAA, with very few exceptions is not involved in Military projects, formal certification is not required. Instead, military agencies typically self-certify under the term "Compliance". Thus, militaries require compliance to DO-178/DO-254, not certification. The difference? In Compliance,

The FAA is not involved,

DERs are not required, though they are advised, and

Certain requirements are relaxed, including the ARP-4761-based safety analysis and for DAL D (especially in the USA), formal DO-254 application.

GAP Analysis

Most military organizations and suppliers have established generally high-quality organizations and processes. When adopting DO-178 and DO-254, they can reuse much of their existing processes, documentation, and artifacts. Often, they operate at a 60-70% DO-178 and 30-50% DO-254 adherence level without even considering DO-178. Therefore, when faced with the requirement to "comply with DO-178", it is most cost-effective for them to do a Gap Analysis. This Gap Analysis assesses their current processes and determines the "gaps" compared to full DO-178 (or DO-254) adoption. A Gap Analysis typically takes 2-4 person weeks to perform by experienced DO-178/DO-254 experts, and can save years by maximizing re-use. And, unlike Military Standards which have strict requirements for document/artifact format/content, DO-178/DO-254 provides for greater latitude, hence the retention and reuse of existing items.

When AFuzion performs a DO-178/DO-254 Gap Analysis for a military client, the following levels of "Gaps" are typically found within the audited organizations (where "0% Gap = 100% Compliance"):

SEI CMM Level 1 Organization: Gap = 70% – 90%

SEI CMM Level 2 Organization: Gap = 50% – 75%

SEI CMM Level 3 Organization: Gap = 35% – 60%

SEI CMM Level 4 Organization: Gap = 25% – 40%

SEI CMM Level 5 Organization: Gap = 20% – 35%

There are two surprising facts from the above. First, even for CMMI Level 5 organizations, the gaps are still significant because CMMI does not include such 178C mainstays as two levels of software requirements, tool qualification, structural coverage (statement, DC, MCDC), extreme robustness testing, tool qualification, data/control flow and coupling analysis, etc. The second surprising fact from the above gaps is wide variation in CMMI Level 3 gaps; a gap of 60% for Level 3 only happens because the engineering more resembles Level 1-2 than Level 3 …

On a DO-178/DO-254 individual activity basis, the particular gaps are typically as follows:

CERTIFICATION ACTIVITY

% "GAP"

PSAC 80%

QA Plan 20 - 30%

CM Plan 10 - 20%

Development Plan 40 - 50%

Software Verification Plan 60 - 70%

Safety Assessment 80 - 90%

Requirements Definition 20 - 30%

Design 10 - 15%

Logic/Code 5 - 10%

Functional Test 5 - 10%

Structural Coverage Tests 90 - 100%

CM 10 - 30%

QA/PA 50%

Tool Qualification 100%

Checklists 30 - 50%

Reviews 30 - 50%

Audits 30 - 50%

DER Liaison 100%

DO-254 Benefits on Military Projects.

DO-254 is not free, as cited above. However, DO-254 can be cost-effective, when understood and implemented properly, even on military projects. Why then are so many military organizations adopting DO-178/DO-254? Because there truly are actual benefits. The following describes the most commonly obtained benefits from DO-178/DO-254 for Military projects based upon the experience of this author's success on over 150 aerospace projects:

Greater Supplier Visibility. With DO-178 (and DO-254), the expanded artifact/review processes provide greater supplier visibility.

Greater upfront requirements clarity. DO-254 mandates thorough and detailed requirements/design. Such detail, and the necessary discipline, force answers to be provided up-front instead of being deferred. Assumptions are drastically minimized. Consistency of requirements and their testability is assured. Iterations and rework due to faulty and missing requirements are greatly reduced.

Fewer logic iterations. Logic iterations, or churn, are the bane of software engineering. In many cases, 10, 20, and even 30 versions of evolving code files exist on new products. Nonsense. Logic should be largely correct the first time it is written and should not require dozens of updates to get it right. Logic should be reviewed by analyzing implementation versus documented requirements.

Decreased single-point project failures. Software is an art; artists resist documenting their work and subjecting it to common development standards and peer reviews. Without standards, discipline, and modern software engineering principals, software teams devolve into a group of loosely structured rogue artists; these artists are highly valuable, creative, and talented persons. But, the loss, or deficiency, of any such artist for any reason is catastrophic to the team. Unless their work is documented, understood, and consistently applied as for the other artists. DO-254 greatly reduces the possibility of such single-point project failures.

Improved management awareness of true schedule status. How many software projects report a "99% Complete" status week after week? How is software progress measured? How can management truly ascertain completion status of software? The answer to all these questions is via modern and accurately detailed management techniques built around DO-254. The provision for insight, traceability, and accurate status on design, development, testing, integration, and reviews is found in DO-254.

Greater consistency within software. Software is like a chain: only as strong as its weakest link. Hardware that is 99% correct is 1% incorrect, which means it is unsafe. The weakest hardware module, or hardware engineer, is on the critical path of hardware safety. All hardware must be consistent per its level of criticality and DO-254 enforces such.

Fewer defects found during integration. Integration can be a lengthy iterative process where major defects requiring design changes are revealed and fixed. Not with DO-254, where integration is typically 50-75% faster than non-DO-254 environments.

Improved reusability. Via thorough and consistent documentation required by DO-254, modularization, enforcement of documented modern engineering principles, and reviews to ensure all the above was achieved, re-usability is greatly improved. In hardware (and software), re-usability is the holy-grail. But the reality is that unless a hardware component is at least 80% re-usable (e.g. unchanged), then it is quicker and less risky to simply start from scratch. And most hardware is less than 50% "reusable". With DO-254, and enforcement of design/logic standards, coupled with independent reviews and traceability, most modules should be at least 90% reusable.

Easier regression testing. You build your product on the assumption that it will be successful, and therefore have a long life. And you know your software will evolve through new applications, installations, and versions. Regression testing can be expensive if extensive or manual; DO-254 provides thorough traceability to determine which modules need changes or analysis, and corresponding retesting.

Improved hardware/software integration. Integrating software onto its target hardware is typically challenging for embedded systems: the development environment is quite removed from the target environment and implemented via different engineering teams. DO-178C mandates that software components which could be affected by hardware be tested on the hardware, e.g. hardware/software interfaces, interrupts, timing, board-level components, BSP/RTOS, etc. And the improved determinism and quality of all these components via DO-178 belies improved hardware integration.

Improved System/Safety Focus. Military projects increasingly follow civil aviation's mandate to consider ARP-4754A for System considerations and ARP-4761A for Safety. By applying the Systems/Safety requirements up front then continuously throughout the project, the overall system development and safety adherence are improved.

For free DO-254 Training whitepapers, see http://afuzion.com/avionics-safety-critical-training-whitepapers/

For DO-254 Training information, see http://afuzion.com/avionics-training/workshops/avionics-hardware-intermediate-do-254-training-class/