"You think me an old fogey and an old tory": Gazing on the royal form, Mr. Deasy continues to wage his culture wars between Catholics and Protestants, shifting the ground now from money to politics. Tory was another name for the Conservatives in the British Parliament, the party that generally supported the status quo and resisted major social reforms—including Home Rule for Ireland.
In the 1870s and 80s, after the collapse of the Fenian movement, momentum shifted toward parliamentary action, as the Home Rule league sought to finish O'Connell's work by repealing the Act of Union passed in 1800, restoring an Irish Parliament in Dublin, and creating a measure of Irish self-rule within the British empire. It was seen as a centrist course between the extremes of Irish insurrection and British imperialism, and it brought together Irish leaders like Charles Stewart Parnell and English ones like William Ewart Gladstone, leader of the Liberals in Parliament and Prime Minister for much of the 70s, 80s, and 90s.
Like English voters, Irish voters elected Liberal and Conservative representatives to Parliament, but starting in the 1870s they also increasingly elected members of what was successively known as the Home Rule League, the Nationalist Party, and the Irish Parliamentary Party (IPP), who strongly advocated for self-rule, and Unionists, who strongly opposed it. The former cause proved immensely stronger. In the 1885 elections the IPP won 85 of the 103 Irish seats. After Parnell's scandal and downfall in 1890, the party fragmented and the Home Rule movement collapsed.
Mr. Deasy seems to be using "tory" as a purported description of himself much as he uses "fenian" at the end of the paragraph to characterize Stephen—i.e., as a broad cultural generalization rather than a specific claim of affiliation. Deasy may or may not vote Conservative. Stephen and his father Simon may converse with revolutionaries, but both are Parnellites, not bomb-throwers.