A service that worked perfectly on a test bench suddenly refuses to start on a hardened RHEL 9 box: the log talks about permission denied where all Unix permissions look fine, and the words Security-Enhanced Linux appear in every second search result. That moment is when most administrators first meet SELinux in enforcing mode, and the meeting rarely feels friendly. The good news is that on RHEL 9 the whole workflow, from a cryptic denial to a working custom policy module, fits into a repeatable evening procedure. This article walks that route end to end: reading denials properly with ausearch, understanding the reason behind each one with audit2why, generating and hand-editing policy modules, and shipping them without turning the server into a permissive exception on the network.
What enforcing mode actually blocks and how to see it
SELinux is a mandatory access control layer that sits below the usual file permissions and decides whether a labeled process may perform an operation on a labeled object. When the answer is no, the kernel writes an AVC denial into the audit log and, in enforcing mode, blocks the operation outright. Nothing in the classic permission model hints at the reason: the file can be world-readable, the directory executable, the service running as root, and the syscall still fails, because the label of the process is not allowed to touch the label of the file in that particular way.
The first commands worth memorizing belong to the status family. getenforce prints the current mode: Enforcing, Permissive or Disabled. sestatus adds context, including the policy name, which on a stock RHEL 9 is targeted, and the enforcing status. Both are read-only and safe on any system, production included:
getenforce Enforcing # sestatus | head -5 SELinux status: enabled SELinuxfs mount: /sys/fs/selinux Loaded policy name: targeted Current mode: enforcing
Labels follow the syntax user:role:type:level. For everyday work the third field, the type, is the one that matters, and the -Z flag shows it everywhere: ls -Z on files, ps -Z on processes, id -Z on the current shell. A quick look at two labels already tells half the story, because the targeted policy grants permissions type to type, and a denial almost always means the process type lacks a rule for the file type.
Denials land in /var/log/audit/audit.log when the audit service runs, and in the kernel ring buffer otherwise. Every denial carries the operation class and the labels of both sides. Reading raw AVC lines is a skill, but RHEL ships tools that make the reading optional, and those tools are the subject of the next section.
Reading denials with ausearch before jumping to conclusions
The ausearch tool queries the audit log with filters that actually help. The two most used invocations look like this:
ausearch -m AVC,user_avc -ts recent # ausearch -m AVC,user_avc -ts today
The -m flag selects message types, AVC being the denial itself and user_avc its userspace cousin that SELinux-aware applications emit. The -ts flag bounds the search in time: recent means the last ten minutes, today means since midnight, and a precise value like -ts 14:30 also works. The output groups every denial into a numbered record, which turns a wall of noise into a list of distinct incidents, one record per access attempt, with the timestamp preserved.
Two habits separate a useful session from a lost one. First, look for the denied operation, not just the labels: open, read, write, connect and name_bind tell completely different stories about what the application was attempting. Second, count how many distinct denials exist before fixing anything, because fixing number one often rewrites numbers two through five into something else, and a batch fix built from a stale log produces a policy for a problem that no longer exists.
When the audit service is missing, denials still appear through dmesg, and ausearch can read them from a file: ausearch -m AVC -i -if /path/to/log. On systems where denials seem absent while the failure persists, the dontaudit rules are the usual suspects, and the closing section of this article covers how to reveal them properly.
From denial to decision with audit2why
Before writing any rule, ask why the denial happened. audit2why answers that question by interpreting each AVC line and classifying the reason. The classic pipeline reads the audit log straight into it:
ausearch -m AVC -ts today | audit2why
Every interpreted denial ends with one of three verdicts. The first and most valuable one names a boolean: the output says that the denial would be allowed by an existing policy switch, something like httpd_can_network_connect, and the fix is one command away. The second verdict says was not allowed, meaning no rule exists for this access pair and a custom module is needed. The third says constraint violation, which points at MLS or MCS constraints rather than missing type rules, and those need a different conversation about the security model, not about allow rules.
Booleans deserve their own paragraph, because they are the difference between a hardened policy and a hand-rolled one. The targeted policy ships with hundreds of prewritten switches, each enabling a whole class of behavior that the default considers risky. Listing them with semanage boolean -l or getsebool -a shows the on and off states together with a one-line description, and the httpd_can_network_connect example is the canonical teaching case: a web application that must connect to a database backend needs exactly this boolean, not a custom module. The change is immediate and survives reboots when made with the -P flag:
setsebool -P httpd_can_network_connect on
The habit that saves weekends is simple: run audit2why before audit2allow. If the verdict names a boolean, flip it and never write a line of policy. Only a genuine was not allowed verdict justifies the next section.
Generating and understanding modules with audit2allow
When a denial genuinely lacks a rule, audit2allow turns the denial into a draft module. The name contains allow, and that is a warning in itself: the tool generates exactly what the denial asked for, with no judgment about whether the access is safe. Feeding it the whole audit log produces a policy that allows everything that ever failed, which is why the standard workflow pipes a filtered, reviewed selection rather than the raw log:
ausearch -m AVC -ts today | audit2allow -M mywebapp
The command writes three files: mywebapp.te with the human-readable policy source, mywebapp.mod with the compiled intermediate form, and mywebapp.pp, the installable package. The .pp file is what the kernel consumes, but the .te file is what a professional reads, audits and edits before anything reaches production. A generated .te typically looks like this:
module mywebapp 1.0;
require {
type httpd_t;
type myapp_port_t;
class tcp_socket name_connect;
}
#============= httpd_t ==============
allow httpd_t myapp_port_t:tcp_socket name_connect;
The require block lists the types the module depends on, and the allow lines grant precisely the access that was denied. This is the moment to check three things. Does the module grant only the access actually needed, or did one noisy application drag in unrelated denials? Are the type names the ones the application should be using, or did a mislabeled file generate a rule that fixes a labeling bug the wrong way? Would a boolean, checked in the previous section, cover the same ground? A module that passes those three questions is worth installing:
semodule -i mywebapp.pp # semodule -l | grep mywebapp mywebapp
Uninstalling is just as easy, semodule -r mywebapp, which makes policy modules reversible and encourages shipping them as versioned artifacts rather than as folklore on a single server.
Hand-written policy and file context management
Generated modules cover allow rules, but half of real SELinux work is labeling. When an application reads from a nonstandard directory, the correct fix is usually not to allow access to a wrong label but to give the directory the right label. The semanage tool manages persistent label rules that survive relabeling, unlike a manual chcon that dies at the next autorelabel:
semanage fcontext -a -t httpd_sys_content_t "/webdata(/.*)?" # restorecon -Rv /webdata
The first command registers a regular expression mapping every path under /webdata to the httpd_sys_content_t type, and restorecon applies the stored rules to the actual files, recursively and verbosely. Checking what rule applies to a path without touching it is matchpathcon /webdata/index.html, and listing all custom contexts is semanage fcontext -l. Ports follow the same pattern: semanage port -a -t http_port_t -p tcp 8080 teaches the policy that a custom port belongs to the web server class, which fixes the classic bind failure of a renamed web server listener.
For applications with no existing type, a fully hand-written module starts from a .te file with the same module and require skeleton, written from the documentation of the policy interface. The sepolicy tool generates starting points, sepolicy generate --init /usr/local/bin/myapp produces a complete package with a .te, an interface file and a spec file for RPM packaging, which is how a homegrown service becomes a first-class SELinux citizen instead of an exception. Hand-written does not mean heroic: most production modules are ten lines, one require block, two allow rules and a comment explaining why the rule exists, because a policy rule without a reason attached turns into archaeology within a year.
Rebuilding without dontaudit, shipping and the field checklist
Some denials are deliberately silenced by dontaudit rules that ship with the policy, which makes a failure invisible in the log and much harder to reason about. The standard diagnosis move is rebuilding the policy with dontaudit disabled: semodule -DB disables every dontaudit rule and rebuilds the policy store in one step, and after reproducing the failure the flood of previously hidden denials explains what the application was trying to do. Re-enabling afterwards is semodule -B, which restores the shipped state. Leaving dontaudit off globally on a production box is a bad idea, because the noise level becomes deafening; the switch exists for a diagnosis session, not for a lifestyle.
Shipping policy follows the same logic as shipping code. The .te source is the artifact under version control, the .pp package is the build output, and the deployment runs through the same channels as any other configuration change, whether that is an RPM, an Ansible task with the selinux module, or a plain semodule -i in a tested runbook. A module tested on staging and rolled out with a known rollback command carries none of the fear that made administrators disable SELinux for two decades.
The field checklist that closes the loop is short. Reproduce the failure and collect denials with ausearch -m AVC -ts recent. Feed them to audit2why and check for a boolean verdict first. Flip the boolean if one exists, and only otherwise generate a module with audit2allow. Read the .te file, trim what the application does not need, install with semodule -i and verify the fix. For path problems, prefer semanage fcontext plus restorecon over any allow rule. Record every custom decision in the runbook, because an enforcing-mode server with an undocumented policy is a puzzle box for whoever inherits the pager.
Per-domain permissive is the middle path worth knowing about: semanage permissive -a myapp_t turns enforcing off for one type only, letting the application run freely while every denial keeps logging into the audit trail. It is the right tool for staging a brand new service whose policy does not exist yet, because the application works while the log quietly accumulates the raw material for its future module. The inverse command, semanage permissive -d myapp_t, returns the type to enforcing once the policy is ready. Used this way, permissive domains are a staging tool with a built-in exit, and they beat disabling SELinux globally the way a scalpel beats a hammer.