General approach
The records and approach to researching a man of the cavalry is no different to that of a man who served, for example, in the infantry or artillery. The men’s service records, campaign medal records, gallantry and bravery awards, casualty lists, records of death, etc are all in the same collections.
See my pages How to research a soldier and A Soldier’s life for full details.
Records specific to the cavalry
Online
Ancestry has a digitised set of the records of service of men who served with the Household Cavalry (that is, the 1st Life Guards, 2nd Life Guards and Royal Horse Guards) in a collection which covers 1799 to 1920. The originals from which the digital copies have been produced are from the National Archives WO400 collection. Ancestry has however placed this collection within its Fold3 portfolio.
The contents of each man’s file varies but is essentially similar to that held in the WO363 and WO374 series for other regiments and corps, but in many cases is stripped down to little more than his attestation form.
Print and physical copy
The National Army Museum, the Imperial War Museum and the Liddle Collection at the University of Leeds all have holdings that cover various aspects of the men and units of the cavalry, but unless you are exceptionally lucky you are not likely to find your man mentioned by name.
The Yeomanry regiments, being units of the the Territorial Force, are often well represented in collections in local and county archives, arising from the way they were locally funded, accommodated and managed. Especially for men who served in these units before the Great War in in its early months, the local archives can be valuable for your research.
I havelrather lost track of the Household Cavalry Archives, but they used to have a copy of their men’s records. I presume that they still do. Perhaps worth an email to establish what they might hold for a given soldier. Details at this link