Bloom thinks that the "August bank holiday" might be a good time to pay a visit to Milly in Mullingar, because he would have a three-day weekend. Bank holidays are days when the banks are closed, and some of them, including the first Monday in August, coincide with public holidays when other businesses close down. Irish people refer to public holidays colloquially as bank holidays, as Molly may be doing when she thinks, "not a bank holiday."
In 1871 the Bank Holidays Act in the UK Parliament established Easter Monday, Whit Monday (just after Pentecost, which is the seventh Sunday after Easter), St. Stephen's Day (or Boxing Day, just after Christmas), and the first Monday in August as bank holidays. Banks were already closed on other important holidays like Good Friday and Christmas, and the Bank Holiday Act of 1903, introduced by an Irish MP, added St. Patrick's Day (March 17). So by 1904 there were many bank holidays, not all of them "bank holidays" in the popular sense of general public holidays.
Gifford notes that the August bank holiday is a "long holiday weekend not unlike Labor Day weekend in the United States." It also is an oasis in a desert, with no other holidays for a couple of months before and after. So it readily suggests itself to Bloom as a good time for a late-summer getaway from the city.