BALDOCK, Circuit Judge.
Defendants appeal the district court’s partial denial , of their motion to dismiss Plaintiff’s complaint for failure to state a claim upon which relief can be granted. Defendants argue that the district court erred by (1) refusing to convert their Fed.R.Civ.P. 12(b)(6) motion to dismiss to a Fed.R.Civ.P. 56 motion for summary judgment and (2) denying their defense of qualified immunity.
Plaintiff is the former police chief of the Town of Fairland, Oklahoma. She was terminated from her position on April 6, 1995. The individual Defendants were members of the Fairland Board of Trustees when Plaintiff was terminated. Plaintiff asserts that Defendants- violated her First Amendment right to free speech by terminating her employment because she criticized, and recommended the termination of, a police officer, and that Defendants violated the Fourteenth Amendment Equal Protection Clause by terminating her on the basis of gender. She also claims that Defendants violated her Fourteenth Amendment due process rights by terminating her without cause and without a hearing.
In their motion to dismiss, Defendants argued that they were entitled to qualified immunity only as to Plaintiffs Fourteenth Amendment due process claim. Plaintiff, however, in her response to the motion to dismiss, addressed the qualified immunity defense in regard to all three of her federal claims. Then, in Defendants’ reply to Plaintiffs response to the motion to dismiss, Defendants argued, albeit in a cursory manner, the defense, of qualified immunity in regard to Plaintiffs First .Amendment and Fourteenth Amendment equal protection claims. Defendants had also raised the defense of qualified immunity in their respective answers. In support of the motion to dismiss, Defendants submitted evidentiary material, which the district court refused to consider. The district court granted the motion to dismiss in part and denied it in part. The district court dismissed Plaintiffs Fourteenth Amendment due process claim for failure to state a claim, and, in the alternative, concluded that Defendants were entitled to qualified immunity as to this claim. The district court also concluded that Plaintiffs complaint stated a First Amendment claim and a Fourteenth Amendment equal protection claim. The district court did not address qualified immunity regarding these two claims.
I.
Although an order denying a motion to dismiss is generally not immediately appeal-able, an order denying qualified immunity is immediately appealable as a “final decision” under 28 U.S.C. § 1291. Mitchell v. Forsyth, 472 U.S. 511, 530, 105 S.Ct. 2806, 2817-18, 86 L.Ed.2d 411 (1985); see also Johnson et al. v. Jones, 515 U.S. 304, 319-20, 115 S.Ct. 2151, 2159, 132 L.Ed.2d 238 (1995). An interlocutory appeal of the denial of qualified immunity is justified because, if granted, qualified immunity shields a defendant from the “burdens of litigation” and frees him from standing trial. Behrens v. Pelletier, 516 U.S. 299, 306, 116 S.Ct. 834, 838-39, 133 L.Ed.2d 773 (1996). Consequently, when the qualified immunity defense is raised, the defendant should be spared the burden of proceeding with the litigation until the plaintiff establishes that the defendant’s alleged actions violated clearly established law. Gallegos v. City & County of Denver, 984 F.2d 358, 361-62 (10th Cir.1993).
In this case, Defendants raised the defense of qualified immunity, but the district court did not address it in regard to two of Plaintiffs federal claims. The parties assume that the district court denied qualified immunity as to all the federal claims. However, a review of the district court’s order reveals that the court did not address qualified immunity in regard to Plaintiffs First Amendment and equal protection claims. Because the district court did not deny qualified immunity, we may lack jurisdiction over this interlocutory appeal. Although the parties have not raised this issue, we have an “independent duty” to inquire into our jurisdiction. See Phelps p. Hamilton, 122 F.3d 1309, 1315-16 (10th Cir.1997). We must, therefore, address sua sponte whether we have jurisdiction over this appeal.
We have previously concluded that we have jurisdiction over an appeal from an order postponing a decision on qualified immunity. Workman v. Jordan, 958 F.2d 332, 336 (10th Cir.1992). In Workman, we reasoned that unless such orders are immediately appealable, a defendant loses his right to be free from the burdens of discovery and trial. Other circuits have concluded that orders failing or refusing to consider qualified immunity are also immediately appealable. Helton v. Clements, 787 F.2d 1016, 1017 (5th Cir.1986); Craft v. Wipf, 810 F.2d 170, 173 (8th Cir.1987); Musso v. Hourigan, 836 F.2d 736, 741 (2nd Cir.1988) (denial of summary judgment motion immediately appealable even though district court failed to address qualified immunity defense). We agree with this approach. Regardless of whether a district court merely postpones its ruling or simply does not rule on the qualified immunity defense, if we deny appellate review, a defendant loses the right not to stand trial. See Workman, 958 F.2d at 336. Accordingly, we may properly exercise jurisdiction over this appeal.
II.
We now turn to the merits. First, we must determine whether the district court abused its discretion by refusing to convert Defendants’ motion to dismiss into a motion for summary judgment. A motion to dismiss for failure to state a claim upon which relief can be granted must be converted into a motion for summary judgment whenever the district court considers matters outside the pleadings. Fed.R.Civ.P. 12(b)(6). As Defendants recognize, courts have broad discretion in determining whether or not to accept materials beyond the pleadings. 5A Charles Alan Wright & Arthur R. Miller, Federal Practice and Procedure § 1366 (1990). Reversible error may occur, however, if the district court considers matters outside the pleadings but fails to convert the motion to dismiss into a motion for summary judgment. Miller v. Glanz, 948 F.2d 1562, 1565 (10th Cir.1991). Such error is harmless if the dismissal can be justified under Fed.R.Civ.P. 12(b)(6) standards without consideration of the matters outside the pleadings. Id. at 1566.
In the present case, the district court stated that it was not considering matters outside the pleadings and therefore would not convert the motion to dismiss into a motion for summary judgment. Defendants argue that the district court did in fact consider matters outside the pleadings. In support of this assertion, Defendants point to the following statements in the district court’s order: (1) that the Town of Fairland had a “board of trustees form of government”; (2) “that the actions taken by the individual members of the Board of Trustees in terminating the Plaintiff ... were based upon legal advice in regai-d to the then existing law of the State of Oklahoma and of the United States”; (3) that Plaintiff was an “at-will” employee; and (4) that Plaintiff made statements regarding police officer Bill Pinion’s actions and that she placed a termination recommendation on the Board’s agenda. Defendants argue that these facts were gleaned from the affidavit of Robert James and Plaintiffs deposition. Our review of the record reveals otherwise. These allegations were also contained in the pleadings. Thus, Defendants have not demonstrated that the district court relied on matters outside of the pleadings when ruling on the motion to dismiss. Accordingly, we conclude that the district court did not abuse its discretion in failing to convert Defendants’ Fed.R.Civ.P. 12(b)(6) motion into a motion for summary judgment.
Defendants also argue that they are entitled to qualified immunity as to Plaintiffs two remaining federal claims. We decline to decide the issue of qualified immunity because the district court failed to consider it. As a general rule an appellate court does not consider an issue not passed upon below. Workman, 958 F.2d at 337. Accordingly, this matter will be remanded to the district court for a proper determination in the first instance of whether Defendants are entitled to qualified immunity as to Plaintiffs First Amendment and equal protection claims.
AFFIRMED IN PART and REMANDED for further proceedings consistent with this opinion.
. We need not address the district courts treatment of Plaintiffs state law claims as those claims are irrelevant to the issues in this appeal.
. Plaintiff does, not argue that Defendants waived the defense of qualified immunity by failing to address it in their motion to dismiss.
. Because the district court granted Defendants motion to dismiss in regard to Plaintiff’s Fourteenth Amendment due process claim, we interpret the appeal as challenging only the district court’s treatment of Plaintiffs First Amendment and equal protection claims.
. Perhaps the parties erroneously rely on a section of the district court’s order titled Defendants Claim of Qualified Immunity Under State Law, in which the district court addressed statutory immunity in regard to Plaintiffs state law claims under the Oklahoma Governmental Tort Claims Act. Okla. Stat. tit. 51, § 152.1 (1991). The district court does reference qualified immunity” instead of statutory immunity.” However, it is clear from the "scope of employment” standard applied by the district court that the court was in fact denying statutory immunity, not qualified immunity. See Okla. Stat. tit. 51, § 152.1 (“state ... and all its employees acting within the scope of their employment ... shall be immune from liability for torts”) (emphasis added).
. The parties and the district court refer at various times to both a motion to dismiss for failure to state a claim upon which relief can be granted and a Fed.R.Civ.P. 12(c) motion for judgment on the pleadings. Because Defendants filed their motion after they filed their answer, their motion may be treated as one for judgment on the pleadings. Regardless, the standard for converting either motion to a motion for summary judgment is the same. See Fed.R.Civ.P. 12(c).