There is Office365/Sharepoint CSOM Caml query way to get all available list items even thay are in folders in particular list and also get the custom fields data with no need to specify the field internal names in the <ViewFields> part of the Caml query.
using (var clientContext = _context.CreateUserClientContextForSPHost()) { if (clientContext != null) { CamlQuery query = new CamlQuery(); query.ViewXml = @"<View Scope='RecursiveAll'> <Query> <Where><Eq><FieldRef Name='FSObjType' /><Value Type='Integer'>0</Value></Eq></Where> </Query> <RowLimit>5000</RowLimit> </View>"; var listItems = clientContext.Web.Lists.GetByTitle("Retention Rules").GetItems(query); clientContext.Load(listItems); clientContext.ExecuteQuery(); ...
The key component here is
<FieldRef Name='FSObjType' /><Value Type='Integer'>0</Value>
without it the query would not retrieve data for the custom fields.