407. Topological Phase Transitions in Twisted Bilayer Graphene/hBN from Interlayer Coupling and Substrate Potentials

Introduction

Twisted bilayer graphene (TBG) near the magic-angle regime has emerged as a paradigmatic platform for moiré quantum matter, where narrow bands, strong Coulomb interactions, and topology coexist in a highly tunable setting. When TBG is aligned with hexagonal boron nitride (hBN), the substrate breaks sublattice symmetry and modifies the moiré minibands in a way that can stabilize topological flat bands with nonzero Chern number. These bands underlie a broad family of correlated phases, including integer and fractional Chern insulators, orbital ferromagnets, and anomalous Hall states. A central question is how the topology of the lowest moiré band evolves as microscopic model parameters are varied, especially the interlayer tunneling amplitudes and the hBN-induced potentials.

In continuum descriptions of TBG/hBN, the electronic structure is governed by a competition between interlayer coupling, twist-angle geometry, and substrate-induced perturbations. Two interlayer tunneling amplitudes, conventionally denoted w0 and w1, distinguish AA- and AB/BA-type tunneling processes. Their ratio is known to strongly affect the flatness, bandwidth, and wavefunction structure of the moiré bands. At the same time, hBN contributes both a staggered sublattice potential and an additional moiré-scale periodic modulation. These terms can open gaps, shift band inversions, and alter Berry curvature distributions, thereby changing the Chern number of isolated bands.

A systematic mapping of these effects is important for both theory and experiment. Experimental observations of quantized anomalous Hall responses and fragile-to-topological band rearrangements suggest that the moiré band topology is not fixed by twist angle alone, but is highly sensitive to substrate alignment and relaxation-induced tunneling anisotropies. A comprehensive phase diagram in the space of w0, w1, and hBN potentials therefore provides a framework for interpreting observed topological states and predicting new ones.

Continuum Model and Topological Invariants

The low-energy continuum model of TBG/hBN is built from Dirac Hamiltonians for each graphene layer coupled by moiré-periodic tunneling. For valley K, the Hamiltonian contains intralayer Dirac terms rotated by the twist angle and interlayer tunneling matrices with two distinct amplitudes. The parameter w1 controls AB/BA tunneling, while w0 captures AA tunneling and is often reduced relative to w1 because of lattice relaxation. This asymmetry is crucial: the limit w0 = w1 tends to produce less isolated flat bands, whereas a reduced w0 can enhance band topology and band isolation.

The hBN substrate enters in two ways. First, a staggered sublattice potential acts as a mass term that breaks inversion and sublattice symmetry in graphene. Second, the hBN lattice can generate an additional moiré potential with the same long wavelength as the TBG moiré pattern, depending on alignment and relaxation conditions. Together, these terms can lift degeneracies at high-symmetry points, deform the Berry curvature landscape, and trigger topological transitions.

The topological character of an isolated moiré band is quantified by its Chern number,
C = (1/2π) ∫BZ d^2k Ω(k),
where Ω(k) is the Berry curvature. In systems with C3 rotational symmetry, band inversions can occur at generic momentum points related by symmetry, as well as at high-symmetry points such as Γ, K, and M. When a gap closes and reopens, the integrated Berry curvature can change discontinuously, producing a Chern number transition. The detailed mechanism depends on which degeneracy is lifted and how the Berry curvature is redistributed during the process.

Parameter Space of Interlayer Coupling and Substrate Potentials

A key result of the systematic study is that the topology of the lowest flat band is not controlled by a single parameter, but by a multidimensional interplay among w0, w1, and the hBN-induced terms. Varying w0 and w1 changes the degree of band hybridization and the relative importance of interlayer coherence versus localization. In practice, increasing w0 tends to enhance AA hybridization and can destabilize certain topological phases, while reducing w0 often favors isolated bands with nontrivial Chern number. The parameter w1, by contrast, affects the primary momentum-space coupling between Dirac cones and therefore influences the overall moiré band dispersion and the location of gap closings.

The staggered hBN potential acts as a symmetry-breaking mass that competes with the twist-induced Dirac structure. At weak substrate coupling, the moiré bands may remain topologically trivial or host low-Chern phases. As the staggered potential grows, it can drive the system through successive topological transitions by shifting the relative ordering of conduction and valence minibands. When the hBN moiré potential is included, the phase diagram becomes even richer: additional periodic modulation generates extra avoided crossings and band inversions, leading to more intricate regions with higher Chern number.

The resulting phase diagrams reveal a progressive enrichment of the topological landscape. Instead of a simple sequence of trivial and C = ±1 phases, the system supports multiple topological sectors, including C = 2, 3, 4, and 5 states in experimentally relevant regions of parameter space. These high-Chern phases are not accidental: they emerge from repeated band inversions and symmetry-constrained touchings that transfer Berry curvature among adjacent bands.

Band-Inversion Mechanisms

The topological transitions can be classified according to the momentum-space mechanism by which the gap closes. Three main scenarios appear prominently.

First, gap closings may occur at generic C3-symmetric k points. Because the moiré system preserves threefold rotation symmetry, a band inversion at one generic point is accompanied by symmetry-related inversions at the rotated points. Such transitions often involve Dirac-like crossings that transfer a quantized Berry flux between bands. The associated Chern number change is determined by the chirality of the touching and the number of symmetry-related nodes.

Second, transitions can occur at high-symmetry momenta such as Γ or K. At these points, degeneracies may be protected or nearly protected by the combined action of symmetry and band structure. When the substrate potential or tunneling asymmetry shifts the relative band energies, a high-symmetry crossing can occur, followed by a reopening with altered band ordering. These transitions are particularly important near phase boundaries separating low-Chern and high-Chern regimes.

Third, parabolic touchings can mediate topology changes. Unlike simple linear Dirac crossings, parabolic band touchings involve higher-order dispersion and can produce more abrupt rearrangements of Berry curvature. In moiré systems with strong coupling and substrate perturbations, such quadratic band touchings can split into multiple Dirac points or gap out in a way that changes the Chern number by larger integers. This mechanism is especially relevant for the emergence of C = 3, 4, and 5 states.

In all three cases, the key diagnostic is the redistribution of Berry curvature near the closing point. As the gap narrows, the curvature becomes sharply concentrated in momentum space. After the transition, the curvature may remain localized near the same region or migrate to symmetry-related sectors of the Brillouin zone, reflecting the new topological character of the band.

Berry Curvature Evolution

The Berry curvature provides a direct window into the microscopic origin of the topological transitions. In trivial or weakly topological phases, Ω(k) is typically distributed in a relatively smooth manner across the moiré Brillouin zone. As the system approaches a band inversion, curvature hot spots intensify near the closing momentum. This concentration signals the transfer of topological charge between bands.

For phases with small Chern number, the Berry curvature often shows a dominant lobe near one set of symmetry-related points. In contrast, high-Chern states arise when multiple hot spots contribute constructively to the total integral. The curvature pattern can become highly structured, with alternating positive and negative regions whose net sum yields the observed Chern number. This is particularly evident when the hBN moiré potential is included, since it introduces additional momentum-space modulation that fragments the Berry curvature into several localized features.

A notable feature of the phase diagrams is that the Berry curvature evolution mirrors the underlying mechanism of the transition. Generic C3-related inversions produce curvature redistribution among three symmetry-equivalent regions. High-symmetry transitions lead to a more concentrated curvature transfer at the relevant point. Parabolic touchings generate broader curvature rearrangements and can produce larger jumps in Chern number. Thus, the curvature profile serves as a fingerprint of the topological transition pathway.

High-Chern Number States

One of the most striking outcomes is the appearance of multiple high-Chern number bands, including C = 3, 4, and 5. Such states are of particular interest because they can support exotic correlated phases when partially filled, including higher-order fractional Chern insulators with richer anyonic content and larger quantized Hall responses. In the moiré context, high Chern numbers are unusual because they require a sequence of band inversions or the accumulation of several symmetry-related Berry-flux contributions.

The emergence of these phases can be understood as follows. Reduced w0 enhances the isolation and winding structure of the flat band, while w1 sets the dominant hybridization scale. The hBN staggered potential then breaks symmetry and can push the system through successive inversions. When the hBN moiré potential is active, additional avoided crossings appear, enabling further accumulation of Berry curvature. The outcome is a topological hierarchy in which the band Chern number increases stepwise as the parameters are tuned.

These high-Chern regions are not merely theoretical curiosities. They occupy finite, contiguous domains in parameter space, suggesting that they may be accessible in real devices where relaxation, strain, and substrate alignment naturally vary. Their presence broadens the landscape of possible correlated states and indicates that TBG/hBN is capable of realizing far more than the commonly discussed C = ±1 phases.

Physical Interpretation and Experimental Relevance

The phase diagrams provide a useful interpretive tool for experiments on aligned TBG/hBN. In real devices, the effective values of w0 and w1 depend on lattice relaxation, pressure, twist-angle inhomogeneity, and local strain. Similarly, the hBN-induced staggered mass and moiré potential depend on the exact alignment angle and dielectric environment. This means that different samples, even at nominally similar twist angles, may realize distinct topological phases.

Experimentally observed quantized anomalous Hall states and insulating phases at integer fillings can be understood in terms of the underlying band topology. A nonzero Chern number in the active moiré band can produce chiral edge states and a Hall response without an external magnetic field. When interactions are included, partially filled Chern bands may stabilize ferromagnetic or fractionalized states. The present analysis clarifies how such phases may arise from the microscopic tuning of interlayer tunneling and substrate potentials.

The work also helps explain why some devices show robust topological signatures while others do not. If the effective parameters place the system near a phase boundary, small perturbations can change the Chern number or destroy band isolation. Conversely, if the device lies deep inside a high-Chern region, the topological character should be more robust. This sensitivity emphasizes the importance of accurate modeling of relaxation and substrate effects in quantitative comparisons with experiment.

Implications for Correlated Moiré Physics

Topological band structure is only the first step toward understanding the many-body physics of TBG/hBN. Once an isolated flat band carries nonzero Chern number, electron-electron interactions can produce a wide range of collective states. The interplay of topology and correlation can favor spontaneous symmetry breaking, orbital magnetism, and fractional quantum anomalous Hall phases. High-Chern bands are especially promising because they may support unconventional fractional states with multiple chiral edge modes and novel quasiparticle statistics.

The systematic phase diagrams also suggest that topology and flatness are not independent design criteria. Interlayer coupling and hBN potentials influence both the bandwidth and the Chern number. Therefore, optimizing a device for correlated topological phases requires balancing band isolation, curvature distribution, and interaction strength. The present results provide a roadmap for this optimization by identifying regions where the band remains flat while acquiring a desired topological index.

Moreover, the mechanisms identified here are likely relevant beyond TBG/hBN. Other moiré materials, including twisted transition-metal dichalcogenides and graphene-based heterostructures with different substrates, also feature symmetry-protected band inversions and tunable Berry curvature. The general lesson is that substrate potentials and interlayer tunneling asymmetries can be used as topological control knobs in a broad class of moiré systems.

Conclusion

Topological phase transitions in twisted bilayer graphene aligned with hBN arise from a delicate interplay between interlayer coupling and substrate-induced potentials. By varying the tunneling amplitudes w0 and w1 together with the hBN staggered mass and moiré modulation, one obtains a rich set of Chern number phase diagrams with multiple topological sectors, including high-Chern states up to C = 5. Each transition is tied to a specific band-inversion mechanism at generic C3-symmetric points, high-symmetry momenta, or parabolic touchings, and these mechanisms are clearly reflected in the evolution of the Berry curvature.

The main message is that the topology of the flat band in TBG/hBN is highly tunable and far more intricate than a minimal model would suggest. The combined influence of interlayer coupling and substrate potentials can progressively enrich the topological landscape, producing a hierarchy of robust topological phases that are relevant to current experiments. These findings provide a theoretical foundation for interpreting anomalous Hall measurements, guiding device engineering, and exploring correlated topological matter in moiré heterostructures.

As experimental control over twist angle, strain, pressure, and alignment continues to improve, the ability to access and manipulate these topological transitions should become increasingly realistic. TBG/hBN thus stands as a versatile platform where microscopic tuning can be used to engineer not only flat bands, but also the topological invariants that govern their many-body physics.