SMS Effectiveness:
How to Build an SMS That Actually Works

The Short Version

If you already have a safety management system, Part 5 doesn't consider you finished. It requires you to measure whether the system is working, find the parts that aren't, and fix them. That obligation has no end date. It runs for as long as you hold the certificate.

Most of the other operators are still wondering whether they will have an SMS by May 28, 2027. You're past that. The question in front of you is different and harder: you have a system, but you don't have a reliable way to know whether it's doing anything.That isn't a gap in your judgment. It's a gap the regulation itself is written to close, but almost nobody reads it that way.

The word "effectiveness" appears in the definition of safety assurance in §5.3, in the management accountability requirement, in the designated-personnel duties in, and in the performance assessment requirement. Effectiveness isn't an upsell that comes after compliance. It's what compliance says.This page walks the requirement section by section, then shows you the specific ways a real, funded, well-intentioned SMS stops producing anything.

Table of Contents

1. What Part 5 requires after your SMS exists

Section 5.3 defines safety assurance as processes within the SMS that function systematically to ensure the performance and effectiveness of safety risk controls, and that the organization meets or exceeds its safety objectives, through the collection, analysis, and assessment of information. The same section defines an SMS as a top-down, organization-wide approach to managing safety risk and assuring the effectiveness of safety risk controls. It defines a safety objective as a measurable goal or desirable outcome related to safety. It defines safety performance as realized or actual accomplishment relative to those objectives.

Those four definitions cover the entire concept of proving your SMS works. Each of those definitions contains specific requirements:

You have to keep it running. §5.5(b) is titled "Continuing requirements", and it says any person required to develop and implement an SMS must maintain it in accordance with Part 5. §5.9(c) says you maintain it as long as you are authorized to operate under Part 135 or §91.147.

You have to be able to prove it on request. §5.9(d) requires you to make available to the Administrator, on request, all necessary information and data demonstrating that you have an SMS meeting the requirements of the part. Not that you had one on the day you declared. That you have one.

Your managers own effectiveness, not your safety manager. §5.23(a)(2)(ii) requires you to define, in your safety policy, the accountability of all members of management for assuring the effectiveness of safety risk controls within their areas of responsibility. §5.25(c)(3) gives designated management personnel the duty to monitor the effectiveness of risk controls. FAA guidance draws the line between those two hard: in AC 120-92D paragraph 3.3.4.3.1, "monitor" means the management representative aids process owners in determining whether risk controls are functioning as designed, and that representative isn't expected to own the SMS or make risk-based decisions. Guidance in paragraph 2.3.4 says managers and process owners responsible for operational processes are also responsible for assuring those areas perform as intended.

Your accountable executive owns performance. §5.25(b)(1) requires the accountable executive to ensure the SMS is properly implemented and is performing across all pertinent areas. §5.25(b)(5) requires the accountable executive to regularly review safety performance and direct the actions necessary to address substandard safety performance in accordance with §5.75.

You have to collect data. §5.71(a) lists eight things your monitoring processes must include at minimum: monitoring of operational processes, monitoring of the operating environment for change, auditing of operational processes and systems, evaluations of the SMS and operational processes and systems, investigations of incidents and accidents, investigations of reports of potential noncompliance with regulatory standards or your own risk controls, a confidential employee reporting system, and investigations of hazard notifications received from outside. §5.71(b) requires processes that actually analyze that data. Note item four. Evaluations of the SMS. The regulation requires the system to be pointed at itself.

You have to assess performance against objectives. §5.73(a) requires assessments of safety performance against your safety objectives, and it specifies that those assessments include reviews by the accountable executive. Those assessments have five jobs: ensure compliance with the risk controls you established, evaluate the performance of the SMS, evaluate the effectiveness of the risk controls established under §5.55(c) and identify any that are ineffective, identify changes in the operating environment that may introduce new hazards, and identify new hazards.

And the loop closes in the regulation. §5.73(b) says that if the assessment turns up ineffective controls or new hazards, you go back through safety risk management. §5.51(d) says the same thing from the other direction: SRM applies to hazards or ineffective risk controls identified through the safety assurance processes. §5.75 requires you to establish and implement processes to correct the safety performance deficiencies your §5.73 assessments identify.

And you keep the evidence for five years. §5.97(b) requires records of the outputs of your safety assurance processes to be retained a minimum of five years. Whatever holds those records is a decision you will live with for a long time. That is the whole obligation. Develop, implement, declare, and then measure the thing forever and fix what the measurement finds. FAA guidance in AC 120-92D paragraph 1.6 adds the constraint that ties it off: scalability lets you adjust how you comply with any section of Part 5, but it does not let you set aside a section. There is no size at which the assessment requirement stops applying.

2. Three things operators with a working SMS believe

You are more likely than most readers of this site to already believe that you're in good shape. That belief usually rests on one of three things, and none of them survives contact with §5.73.

"Our audit came back clean."

An audit is a real product and a good one. It tells you whether the processes you documented exist and whether people are following them. Although that's genuinely useful and worth paying for, it doesn't answer the question §5.73(a)(3) asks. That section is about whether a control is achieving the outcome it was built for, and about naming the ones that are not. A control can be fully documented, correctly implemented, and consistently followed, and still not reduce the risk it was written to reduce. An audit that checks conformance will pass that control every time, because the control is conforming. It just isn't working and the two questions are different. Section 3 is about how to tell them apart.

It's worth knowing what the independent audit data actually showed, back when there was public data. PRISM published annual analyses of ARGUS PROS audits in the early 2010s. Across 84 audits conducted in 2014, the internal evaluation program was the most commonly deficient area at an 81% deficiency probability, with risk assessment and SMS training also above 67%. Across 75 audits conducted in 2013, internal evaluation programs and safety training led, with safety training deficient in 61%. Of the findings from audits conducted in 2012, 58% pointed to risk assessment.

Think about that, there were operators who volunteered to be audited, years before any Part 135 mandate existed. They were typically the most safety-invested slice of the industry, and yet the single most-cited deficiency was the internal evaluation program, which is the part of the system whose job is to check whether the rest of it works.

"We haven't had an accident."

You haven't had an accident yet, and neither have most operators your size in any given decade. Part 135 exposure isn't large enough for the absence of an accident to mean much statistically. A three-aircraft operator flying eight hundred hours a year could run a genuinely dangerous operation for a long time and see nothing.

FAA guidance is unusually direct about this. AC 120-92D paragraph 3.3.1.3.2 says safety objectives should be measurable rather than inspirational, and it names two examples of what not to write. One is a statement about striving to be the best. The other is maintaining a zero-accident rate.

The FAA named a zero-accident rate as an inadequate safety objective in the advisory circular that tells you how to comply with the rule. If that's the standard your program is being judged against internally, you're measuring the one number that will look identical whether your system is excellent or whether you've simply been lucky.

"Effectiveness is the thing consultants sell after compliance."

The market is full of people selling a second engagement on the back of the first, and if a vendor introduces effectiveness as a concept that appears after the declaration is filed, you should be skeptical of them. But go back and read §5.73. It sits in Subpart D of the same rule as everything you already built.

There is no separate effectiveness regulation, no optional appendix, no maturity tier. The assessment requirement was in scope from the day the rule applied to you, and if your implementation project treated it as documentation to be written rather than a process to be run, you have a gap in your compliance, not an opportunity for an upgrade. The declaration you signed said your SMS meets the requirements of Part 5. Subpart D is in Part 5.

3. The two questions that separate a fix from a redesign

This is the most useful thing on this page, but almost no operator has been shown it, even though it's sitting in the section of the rule you already declared compliance with.

Read §5.73(a) as a list of two different jobs rather than a list of five bullets. Paragraph (a)(1) asks whether people complied with the risk controls you established. Paragraphs (a)(2) through (a)(5) ask a separate question: whether the SMS is performing, whether the controls you built are effective, and whether new hazards or environmental changes have appeared.

Now read §5.73(b). It says that if ineffective controls or new hazards are identified under paragraphs (a)(2) through (5), you must use the safety risk management process in Subpart C.Paragraph (a)(1) is not in that list. That means a compliance finding doesn't route back through SRM. It routes to §5.75, which requires processes to correct the safety performance deficiencies your assessments identify.

Two findings, two destinations, written into the rule. Which paragraph your finding sits under determines what you are required to do about it.

FAA guidance describes the same split in plainer language and adds the diagnostic detail the rule leaves out. AC 120-92D paragraphs 2.3.1 and 2.3.2 say that when a negative result shows up, you continue the analysis to determine whether the shortfall is because the controls weren't used as intended, and it offers examples: required training not accomplished, procedures not followed, improper tools or equipment provided.

If the controls weren't used as intended, corrective action is the answer. If the system is being used as intended and still isn't producing the expected results, the guidance says the system design should be reconsidered by going back through safety risk management.

Two branches. Same symptom. Completely different fix.

Branch one, the control was not used as intended. Somebody skipped the step. The training didn't happen. The form was filled out after the fact. The tool wasn't available at the moment of decision. This is a §5.73(a)(1) compliance finding, it goes to §5.75, and the fix is about supervision, availability, or workload.

Branch two, the control was used exactly as designed and the outcome still went wrong. This isn't a discipline problem. This is a §5.73(a)(3) finding of an ineffective control, and §5.73(b) requires it to go back through SRM, because the control itself is what needs to change.

Here is why operators get this wrong so consistently. Branch one is cheap. Retraining, a reminder, a memo, a safety meeting agenda item. Branch two is expensive, because it means going back and rebuilding something that somebody in the organization designed, approved, and is now attached to. When the diagnosis is ambiguous, the pull toward branch one is enormous, and the item closes.

Then it comes back in eight months. That's because there is a compliance dimension to this that is easy to miss. If you route a branch two finding to branch one, you haven't just chosen a weak fix. You've probably skipped a step §5.73(b) requires, and the record will show a closure where an SRM referral should be.

Our recommendation, and this is a recommendation rather than a requirement, is that you add one field to your corrective action record: which paragraph of §5.73(a) does this finding sit under, and what is the evidence. If the answer is (a)(1), the record should say specifically how you know the control was not followed. If it is (a)(3), it should show the SRM referral. An item that cannot answer the question isn't ready to close. The cost of doing this is about ninety seconds per finding. The cost of not having it is that your repeat events are invisible, because every one of them was correctly closed the last time.

4. Are your safety objectives actually measurable?

Everything in §5.73 measures your performance against your safety objectives. Which means if your objectives aren't measurable, the entire safety assurance component has nothing to assess against, and it fails quietly.

Here's the book explanation: §5.21(a)(1) requires your safety policy to include your safety objectives. §5.3 defines a safety objective as a measurable goal or desirable outcome. FAA guidance in AC 120-92D paragraph 3.3.1.3.2 says objectives should be measurable and not just inspirational statements, and that they may be based on key performance indicators or safety performance indicators tracked by the safety assurance component. Paragraph 3.3.1.3.3 says the §5.73 assessment is where decisions get made about whether those objectives were attained. Paragraph 3.3.1.3.5 recommends they be specific, measurable, achievable, relevant, and timely.

All that means is this: go pull your safety policy and read your objectives with one question in mind. At the end of this quarter, could a reasonable person outside your company look at your records and say whether you hit it, without asking you what you meant? If the answer is no, the objective is a value statement. Value statements are fine, but they belong in the code of ethics required by §5.21(a)(7). They aren't objectives, and §5.73 can't assess them.

FAA guidance offers five categories that produce measurable objectives: compliance with regulations, milestones for implementing safety programs or initiatives, reduction of error or incident rates, increased employee involvement through hazard or incident reporting, and tracking of specific safety events such as ground damage, pilot deviations, weight and balance errors, or maintenance errors. The guidance attaches a caution to that last one: don't let event counting pull your focus off the risk factors associated with more serious events.

Paragraph 3.3.1.4.1 suggests focusing safety goals on the top one, two, or three risks from your safety risk profile. The guidance phrases it as focusing on the things that keep you up at night. Two or three real objectives that you assess honestly will do more for you than twelve that nobody reviews. And if you have twelve, the §5.73 assessment becomes a two-hour clerical exercise that everyone learns to dread, which is its own kind of system failure.

5. What to measure, and the false precision trap

Part 5 doesn't tell you what your indicators must be. It tells you to monitor, analyze, assess against objectives, and correct. The specific metrics are yours.That freedom is real, and it's the first thing a vendor will try to take off you.

That's why it's so important to understand that no section of Part 5 names a five by five matrix. FAA guidance in paragraph 3.4.4.3.5 says a risk matrix may be qualitative or quantitative or both, that all process owners need to be trained in whichever tool you define so results stay consistent, and it closes with a plain instruction not to over-engineer the matrix or the risk analysis process.

Part 5 does not use the phrase as low as reasonably practicable (ALARP). Part 5 doesn't contain a section called Management of Change or one called Internal Evaluation Program, though §5.71(a)(3) and (a)(4) require the auditing and evaluation functions an IEP performs, and §5.51 requires SRM whenever you implement a new system, revise an existing one, or develop operational procedures, which is most of what a change process is for.

The failure mode we see most often on the measurement side, and this is our observation rather than a regulatory position, is false precision. An operator builds a dashboard with nine indicators, each rendered to one decimal place, several of which are computed from a sample of four events per quarter. The numbers move but nobody can say whether the movement means anything. Eventually the review becomes a ritual of reading numbers aloud, and the safety meeting stops changing any decision.

Here's a useful test before you adopt an indicator: what would you do differently if this number moved twenty percent? If the honest answer is nothing, the indicator is decoration, and it's costing you the attention of the people in the room. Here are a few things worth measuring at Part 135 scale that most operators aren't:

Reporting rate over time, not reporting count. A count tells you nothing. A rate that falls for two quarters tells you something, and it's usually about trust rather than hazards.

Time from report to acknowledgment. The single strongest driver of whether people file the next one.

Corrective action verification rate. Of the items you closed last year, how many have evidence that somebody went back and confirmed the fix worked.

Repeat findings. The same issue appearing twice is the most honest effectiveness measurement you own.

Branch two rate. Of your findings, what fraction went back through SRM as design problems rather than being closed as compliance problems. If that number is zero across a year, section 3 isn't happening.

There is zero requirement for you to have software to track these. Although software can definitely help depending on the size of the operation, all of these fit in a spreadsheet.

6. Your reporting system is an instrument, and it has a needle

§5.71(a)(7) requires a confidential employee reporting system in which employees can report hazards, issues, concerns, occurrences, and incidents, and propose solutions and safety improvements, without concern of reprisal.

Building that system is a Part 135 SMS implementation task. This section is about what the system tells you once it exists. Reporting volume is not a hazard count. It is a measurement of whether your people believe the system is worth using. Those are different readings and they move in opposite directions from what most managers expect.

An operator whose reports go up hasn't necessarily become more dangerous. On the flip side, an operator whose reports go to zero hasn't become safe. In our experience the second one has usually broken one of three things: people stopped hearing back, somebody got burned once and everybody watched, or the system got harder to use than the workaround. The closure loop is the part that decays first and the part almost nobody measures.

§5.93 requires you to develop and maintain means of communicating safety information, including conveying hazard information relevant to an employee's responsibilities, explaining why safety actions have been taken, and explaining why procedures are introduced or changed. That's how you close the loop.

When somebody files a report and nothing visibly happens, you haven't just lost that report. You've taught everyone who heard about it what filing accomplishes. Confidentiality is genuinely harder in smaller operations and pretending otherwise insults those who work for you. FAA guidance acknowledges this directly in paragraph 3.3.1.4.2: the practicality of confidential reporting is more difficult in a small organization where everybody often knows everybody's business, and if a just reporting culture is not in place, reporting may be limited.

The same paragraph notes that anonymous reporting isn't prohibited, but it costs you the ability to go back to the reporter for more information. Our recommendation for an operation under twenty people is to stop optimizing for anonymity you can't deliver and optimize for demonstrated non-retaliation instead. What protects a reporter in a ten-person company isn't a form. It's the last three times somebody reported something and nothing bad happened to them.

7. How do you know the last fix worked?

Ask an operator with a mature SMS how many of last year's corrective actions were confirmed effective after the fact, and watch what happens. Most can't answer, because the record was never designed to hold that answer. Typically what happens is a problem gets identified. Then a fix gets designed and implemented. The item gets marked closed, but the closure is where the process stops.

§5.73(a)(3) requires you to evaluate the effectiveness of the safety risk controls you established under §5.55(c) and to identify any ineffective controls.

§5.75 requires you to establish and implement processes to correct the safety performance deficiencies the §5.73 assessments identify. A closure without verification satisfies neither of them. It produces a tidy log and no information. There is a related requirement people miss. §5.55(d) requires you to evaluate whether the risk will be acceptable with the proposed control applied, before the control is implemented. So the rule asks you to predict the effect going in and then assess the actual effect coming out. If your records only ever contain the second one, or only the first, you have half of a control loop.

What verification actually looks like at Part 135 scale, and this is our recommendation rather than a requirement, is smaller than people fear:

Write down what you expect to see. At the moment you implement the fix, record what would be true in ninety days if it worked. One sentence. This is the hardest part because it forces you to admit when you don't actually know what the fix will do.

Put a date on the calendar. Not "review at the next safety meeting." A date.

On that date, look at evidence rather than asking whether anyone has complained. The absence of complaints is the same non-measurement as the absence of accidents.

Record the branch. If it didn't work, section 3's question applies. Was the fix not followed, or was the fix wrong. That answer determines whether you are re-closing or referring to SRM under §5.73(b). An operator with two years of verified closures has something no software subscription can produce and no auditor can dismiss: a documented history of the organization finding problems and being able to demonstrate that it solved them. That record is also the thing an inspector under §5.9(d) is most likely to find persuasive, because it is very difficult to fake.

8. The accountable executive review that nobody documented

§5.73(a) requires assessments of safety performance against safety objectives, and it specifies that those assessments include reviews by the accountable executive.

§5.25(b)(5) requires the accountable executive to regularly review safety performance and direct the actions necessary to address substandard performance in accordance with §5.75.

§5.25(c)(5) requires designated management personnel to regularly report to the accountable executive on the performance of the SMS and on any need for improvement.

Three separate sections put your accountable executive inside the assessment loop rather than at the end of a reporting line. However, keep in mind that Part 5 doesn't define "regularly." FAA guidance addresses that gap twice. Paragraph 3.3.3.3.2.3 says the accountable executive must regularly review safety performance, that the interval is not specified, that reviews should occur frequently enough for issues to be identified and corrected in a timely manner, and that there should be documentation showing the review occurred.

Paragraph 3.3.1.3.12 says the same about the policy and objectives review under §5.21(d): there should be evidence in the records that the review has been accomplished. So there is no required cadence but there is a required artifact. Pick your own interval, defend it against the size and complexity of your operation, and be able to produce evidence that the review happened.

Our recommendation is to be honest about who the accountable executive is at your size, because in most operations of ten to fifty people it's the owner, and the owner is running the business. A quarterly review that actually happens and produces two decisions is worth more than a monthly one on the calendar that gets moved.

§5.25(a) sets four tests for the accountable executive and all four are about authority: final authority over operations, control of financial resources, control of human resources, and ultimate responsibility for safety performance. Those tests exist because the review is supposed to be the point in the process where somebody who can spend money looks at a problem and decides. Here is the question to ask at your next one. Not "is the SMS in compliance." Ask what the system caught this quarter that we wouldn't otherwise have found, and what we did about it.

If nobody in the room can name something, that's the finding.

9. What this looks like from the outside

Investigators don't usually discover that a company was blindsided. They discover a record. Reports nobody closed, a procedure that had quietly stopped being used, and a limit somebody decided not to raise. The organization had the information but no process that made anyone act on it.

An SMS that exists only on paper can't make anyone act. Neither can one that exists in practice and is never measured. That second failure is the one worth studying, because it happens to operators who did the work.

On January 29, 2019, a Bell 407 operating as a helicopter air ambulance under Part 135 collided with forested terrain near Zaleski, Ohio. The pilot, flight nurse, and flight paramedic were killed. Two other air ambulance operators had already declined the same patient transport that morning because of the weather.

The NTSB determined the probable cause was the operator's inadequate management of safety, which normalized noncompliance with risk analysis procedures by pilots and operations control specialists and resulted in the flight departing without a comprehensive preflight weather evaluation, leading to an inadvertent encounter with instrument meteorological conditions, failure to maintain altitude, and collision with terrain.

That means the risk analysis procedures existed and the operator had no process. However, the truth is that the process had stopped being followed, and the not-following had become normal. Then look at the procedure itself. Investigators entered the accident flight's conditions into a risk worksheet containing all the components identified in FAA Advisory Circular 135-14B. Scored that way, the flight came out two risk levels higher than the operator's own tool had produced. The operator's risk assessment also didn't require anyone to determine whether another operator had already turned the flight down, which is why the pilot didn't know.

The control was routinely bypassed, and on the occasions it was used, it produced the wrong answer. The instrumentation was dark as well. The director of safety and training had received a single incident report in roughly the year and a half he had held the position, and it didn't concern the Ohio bases.

That's the reading from section 6, and it was available to anyone who thought to look at it as a measurement rather than as a hazard count. And the belief from section 2 was stated out loud. When management considered raising weather minimums in line with industry accreditation guidance, the reasoning against it was that the company had been operating successfully, so why adopt somebody else's standard.

None of this required a bad operator. It required a company that had a safety program, believed it was working because nothing had happened yet, and had no process that would have told it otherwise.

There's one more detail worth knowing. The FAA principal operations inspector assigned to the operator had limited helicopter experience, no rotorcraft rating on his commercial certificate, and no air ambulance experience. His previous inspections had revealed no deficiencies even though the NTSB investigation identified several. External review found nothing wrong right up until the day it mattered, and the operator had every reason to read that as confirmation.

Read more about this tragedy here

10. Did anyone actually learn anything?

Section §5.75 requires you to establish and implement processes to correct the safety performance deficiencies your §5.73 assessments identify. It does not tell you how to know whether a correction took.

Nothing in this section is a Part 5 requirement. This is a judgment from a different discipline, and you should read it that way. Just like in the fighter pilot community, when something didn't go the way it was planned, debrief until you identify the root cause and the corrective action. When it comes to SMS effectiveness, the only thing that matters afterward is whether the organization is going to do it again next week. Here are some things to look for.

The organization can name what changed. Ask what changed after the last significant event. Not what was discussed. What changed. If the answer is a conversation, a reminder, an email to the pilot group, or an item on a safety meeting agenda, then nothing changed. A change has a name, an owner, and a date. Somebody rewrote a procedure. Somebody moved a decision to a different person. Somebody added a piece of information to a form that was not there before. In a fighter pilot debrief the output was never a lesson. It was an instructional fix, assigned to a person, with a deadline, and it was typically recorded on a gradesheet so it could be evaluated on the next flight. If your corrective action file is full of items that closed without producing an artifact somebody could hold up, the file is a record of meetings.

Root cause bottoms out at something a person could do differently. "Pilot error" is not a root cause. Neither is complacency, task saturation, or poor judgment. Those are descriptions of the outcome and when analysis stops there the fix is always the same: tell people to try harder. Unfortunately, that never works, because nobody was trying to fail. In a debrief you keep asking why until you reach something fixable. Not a quality of the person. A decision, a procedure, a piece of information they didn't have, or a piece they had and couldn't use in time. When you hit that, the fix writes itself. When you stop short of it, you get retraining, and the same event comes back in eight months. Open your last three corrective actions and read where the analysis stopped. If all three bottom out at a human quality, your assessment process has a ceiling, and §5.73(a)(3) is asking you to see through it.

Somebody senior has been corrected in front of everyone. In a fighter squadron the flight lead gets debriefed too. Not as a courtesy, but because if rank protects them you'll only ever learn about the mistakes of the lowest-ranking person in the room. So read your findings. If every one of them lands on a line pilot or a line mechanic, and none of them land on scheduling, on dispatch, on maintenance turn times, on the trip somebody said yes to before anyone looked at the weather, you don't have a system that finds problems. You have one that finds subordinates. That is a safety culture finding and it won't show up on any audit checklist, because there's no box for it.

The same thing hasn't come back. Repeat events are the measurement. Everything else is a proxy and when something does come back, that isn't a failure of the system. That's the system giving you data. The question is which of the two things in section 3 happened, and most operators never ask. They just re-close the item. The difference between a working SMS and a filed one is whether that conversation happens on a schedule, with the answer written down, and with someone going back later to see whether it held. You already know how to do this. The regulation just wants it in writing.

11. Start here: what you can check this week

None of this requires a vendor, a purchase, or a project. All of it is reading you already own.

Read Subpart D. Sections 5.71, 5.73, and 5.75 run about a page and a half combined. Most operators with a functioning SMS have never read those three sections consecutively with their own program in front of them.

Pull your safety objectives and apply the quarter test. Could an outsider tell from your records whether you hit them. If not, rewrite two of them this month, don't rewrite all of them at once.

Open your last ten closed corrective actions. Count how many contain evidence that somebody verified the fix after the fact. Whatever that number is, it is your starting baseline for the metric in section 5.

Chart your reporting volume by quarter for the last two years. One chart. If the line is flat at or near zero, that's your first finding under §5.73, not evidence that you have no hazards.

Find the documentation of your last accountable executive safety performance review. FAA guidance says there should be evidence it occurred. If you can't find it in under ten minutes, an inspector won't either.

Pick one recurring problem and run section 3 on it. One issue that has come back more than once. Was the control not followed, or was the control wrong. Write down which and why. That single exercise will tell you more about your program's real maturity than any assessment tool. At the end of it you will know whether your SMS is producing information or producing records, which is the only question this page is about.

12. Frequently asked questions

We already declared compliance. Are we finished?

No. §5.5(b) and §5.9(c) require you to maintain the SMS for as long as you hold the authority, and §5.9(d) requires you to produce data demonstrating it meets Part 5 on request. FAA guidance says validation of SMS performance occurs as part of routine surveillance activities and that areas found deficient are addressed using existing methods for ensuring regulatory compliance. The declaration is the start of the surveillance relationship, not the end of the project.

Not by that name. §5.21(a)(1) requires safety objectives and §5.3 defines an objective as measurable. §5.73(a) requires you to assess performance against those objectives. FAA guidance says objectives may be based on key performance indicators or safety performance indicators tracked by the safety assurance component. So indicators are a common method of meeting the requirement rather than the requirement itself, and you get to choose them.

An audit generally asks whether your documented processes exist and are being followed, which maps to §5.73(a)(1), ensuring compliance with the risk controls you established. An effectiveness review asks whether those controls are producing the intended safety outcome, which maps to §5.73(a)(3). Both are required. They are different questions and a clean result on the first doesn't answer the second.

Part 5 does not specify an interval for §5.73 assessments or for the accountable executive reviews within them. FAA guidance says reviews should occur frequently enough for issues to be identified and corrected in a timely manner, that this varies with the size, scope, and complexity of the operation, and that there should be documentation showing the review occurred. You set the interval, document it, and be able to show you met it.

Your obligation runs to Part 5, whatever tool you use. Third-party certification or acceptance does not substitute for compliance with Part 5, and no platform can conduct your §5.73 assessment for you, because the assessment measures your performance against your objectives and requires reviews by your accountable executive. A platform can hold the records and prompt the process. It cannot be the process.

That is the system operating correctly. §5.73(b) requires you to take ineffective controls and newly identified hazards back through the safety risk management process in Subpart C, and §5.75 requires processes to correct the deficiencies the assessment found. Finding an ineffective control is the intended output of §5.73(a)(3), not a compliance failure.

13. Where to go from here

If you take one thing from this page, make it this: your SMS is required to be able to tell you something you didn't already know. That's the test. Everything else is detail.

Three years out, the operators in good shape won't be the ones who documented the most. They will be the ones whose accountable executive can name the last thing the system caught, whose corrective actions carry verification dates, and who can tell a control nobody followed from a control that was wrong.

None of that can be purchased. It accumulates, which is why an operator who starts measuring this quarter is years ahead of one who starts after a surveillance visit goes badly. The next move is the same from anywhere on that scale. Find out honestly what your system is producing, measured against Subpart D rather than against a vendor's checklist or a clean audit report. If you haven't read §5.71 through §5.75 with your own program in front of you, start there. If you have, work section 11.


SOURCES

14 CFR Part 5, current as of eCFR (checked against the live eCFR text, Title 14 up to date as of 8/17/2026). Sections cited: 5.3 (definitions of hazard, risk control, safety assurance, SMS, safety objective, safety performance), 5.5(a) and (b), 5.9(c) and (d), 5.21(a)(1), 5.21(a)(7), 5.21(d), 5.23(a)(2)(ii), 5.25(a), 5.25(b)(1), 5.25(b)(5), 5.25(c)(3), 5.25(c)(5), 5.51, 5.51(d), 5.55(c), 5.55(d), 5.71(a)(1)-(8), 5.71(b), 5.73(a)(1)-(5), 5.73(b), 5.75, 5.93, 5.97(b).

AC 120-92D, Safety Management Systems for Aviation Service Providers, 5/21/24 (cancels AC 120-92B; any source citing 92B as current is out of date). Paragraphs used: 1.6 (scalability does not permit setting aside sections of Part 5); 2.3.1 and 2.3.2 (the negative-result fork: controls not used as intended versus system not producing expected results, and the corrective action versus SRM redesign paths); 2.3.4 (process owners responsible for assuring their areas perform as intended); 3.2.5.3.4 (validation of SMS performance during routine surveillance; deficiencies addressed using existing compliance methods); 3.3.1.3.2 (objectives measurable not inspirational; zero-accident-rate example; KPIs and SPIs tracked by safety assurance); 3.3.1.3.3 (the §5.73 assessment is where attainment is decided); 3.3.1.3.4 (five categories of objectives and the caution on event counting); 3.3.1.3.5 (SMART); 3.3.1.3.12 (interval for policy review undefined; evidence of review should exist in records); 3.3.1.4.1 (focus on the top one to three risks); 3.3.1.4.2 (confidentiality harder in small organizations; anonymous reporting not prohibited but costs follow-up); 3.3.3.3.2.3 (AE performance review interval undefined; documentation should show it occurred); 3.3.4.3.1 (meaning of coordinate, facilitate, and monitor for designated management personnel); 3.4.4.3.5 (risk matrix qualitative or quantitative, do not over-engineer).

NTSB/AAR-20/01, Helicopter Air Ambulance Collision With Terrain, Survival Flight Inc., Bell 407 Helicopter, N191SF, near Zaleski, Ohio, January 29, 2019. Adopted May 19, 2020. Used throughout §9 for: probable cause language, the two prior operator refusals, the AC 135-14B worksheet scoring comparison, the risk assessment procedure's failure to require a check on prior refusals, the safety director's report volume, management's reasoning on weather minimums, and the principal operations inspector's experience and prior inspection results.

PRISM / ARGUS PROS annual SMS audit analyses, 2012 through 2014, as reported by Flight Safety Foundation and AIN. Used in §2 for deficiency-category figures.

SMS final rule, 89 FR 33068, April 26, 2024. Used for: third-party certification or acceptance does not substitute for Part 5 compliance (FAQ Q5).