Sensitivity of the ATLAS Experiment to Beyond the Standard Model Higgs Interactions in the H->bbgamma Decay

The Standard Model (SM) leaves key phenomena unexplained, motivating searches for physics beyond the SM (BSM). Probing the Higgs sector may reveal insight into new physics. This thesis aims at determining the ATLAS experiment sensitivity to the rare Higgs decay H→bbγ, a process not yet experimentally observed. BSM effects in the Hbbγ interaction are parametrised via Effective Field Theory, where non-zero anomalous couplings modify its cross-section and may enhance sensitivity to the decay.

The analysis is performed in the context of the LHC High-Luminosity phase, using Monte Carlo simulations at sqrt(s)=14 TeV, with an integrated luminosity of 3000 fb^-1 and considering pile-up effects. The decay was studied through the Higgs and W associated production, resulting in a final state with one charged lepton, two b-jets, and a photon. Pre-selection cuts based on the final state are applied to prepare the data for multivariate analysis. Two deep neural network classifiers are trained using different training datasets.

The first method trains solely on the SM signal and background, while the second is trained with combined SM and BSM signal data, plus background. The second method presents better performance overall and is used to estimate the expected signal significance. The results indicate lower sensitivity compared to earlier literature results obtained in similar studies due to the inclusion of more extensive background contributions and pile-up effects.

The evidence of a BSM signal can only be reached for large values of the BSM couplings, close to today's exclusion limits. The study concludes that full detector simulation with detailed pile-up modelling must be performed to design future strategies to deal with the dominant backgrounds of soft and fake photons.