`Exact,' small-sample methods for categorical data are exact in terms of using probability distributions that do not depend on unknown parameters. However, they are conservative inferentially, having actual error probabilities for inference that are bounded above by the nominal level. We examine the conservatism for confidence intervals and survey ways of reducing it, illustrating for estimating and comparing binomial proportions. Fuzzy inference is an adaptation of randomized inference that achieves the error probability exactly. In practice, many would find this approach unsuitable. However, it motivates inferences based on the mid-P value that are less conservative than standard exact methods yet usually approximate well desired error probabilities. We also summarize simple ways of adjusting standard large-sample confidence intervals to improve dramatically their small-sample performance.