My OCR model mislabels section titles as body text. Is a CRF the right fix, or am I overcomplicating it? [P]

Hi everyone, I'm working on extracting the hierarchical structure of long PDF documents (legal/regulatory text, lots of numbered sections) and would like to gather some feedback on my approach before committing to it. What I've done so far: I render each PDF page to an image and run it through Baidu's DeepSeek-OCR model . It returns each detected block with a bounding box [x0, y0, x1, y1] , a label ( title , text , list , table , header , footer , etc.), and the recognized text. The OCR quality itself is genuinely good as the text comes out clean. The problem: the labels can't always be trusted. At this stage I want to extract and detect all the titles in my document, but sometimes a title element gets classified as something else (like normal body text). Concrete example: Say my section has the following hierarchy: ANNEX I — GENERAL PRINCIPLES AND PROCEDURES └── TITLE I — FOREIGN CURRENCY INVESTMENT └── A. Currency distribution └── 1. Redistribution of reserves ├── (a) Introduction │ body text │ list │ ... ├── (b) Procedure for a normal redistribution of reserves │ body text │ list │ ... └── (c) Procedure for an ad hoc redistribution of reserves body text list ... Logically, every element aside from the body text and lists should be detected as title . But the model output is: label='title' x0=475 y0=157 x1=548 width=73 text='ANNEX I' label='text' x0=480 y0=229 x1=542 width=62 text='TITLE I' label='title' x0=334 y0=181 x1=690 width=356 text='GENERAL PRINCIPLES AND PROCEDURES' label='title' x0=407 y0=368 x1=616 width=209 text='A. Currency distribution' label='title' x0=408 y0=392 x1=634 width=226 text='1. Redistribution of reserves' label='title' x0=163 y0=416 x1=304 width=141 text='(a) Introduction' label='title' x0=163 y0=544 x1=578 width=415 text='(b) Procedure for a normal redistribution of reserves' label='title' x0=163 y0=219 x1=586 width=423 text='(c) Procedure for an ad hoc redistribution of reserves' The top-level section marker TITLE I was labeled text , while all the other components were labeled correctly as title . What I'm considering: since I have the text plus features I can derive from the coordinates (indentation/ x0 , centered-vs-left-aligned, line height, vertical gaps, whether the text matches a numbering pattern like A. / 1. / (a) , all-caps, word count, etc.), I was thinking of treating this as a sequence labeling problem and training a CRF (or BiLSTM-CRF) to re-classify each line into title / text / list / table . My questions: Is a CRF a reasonable choice here, or is there a better-suited approach for this kind of layout/structure labeling? Should I consider a GNN approach? Am I overcomplicating this? Would a simpler rule/heuristic system be more robust, given that the numbering is fairly regular? Note #1: this approach should be as general as possible, so that I can reuse it for my other legal documents. Note #2 : titles aren't always in the same horizontal position. Some are centered (e.g. ANNEX I , TITLE I , A. Currency distribution all sit around xc≈511 , the page center), while deeper items like (a) / (b) / (c) are left-aligned at x0=163 . So I can't rely on indentation/ x0 alone to identify or rank titles — a centered title's x0 mostly reflects its text length (a short centered line has a large x0 , a long one a small x0 ), which means raw x0 can even invert the apparent nesting. This is part of why I'm leaning toward a sequence model that combines text + geometry in context rather than a pure indentation rule.