Abstract: Ultra-fine entity typing (UFET) predicts extremely free-formed types (e.g., {\it president, politician}) of a given entity mention (e.g., {\it Joe Biden}) in context. State-of-the-art (SOTA) methods use the cross-encoder (CE) based architecture. CE concatenates a mention (and its context) with each type and feeds the pair into a pretrained language model (PLM) to score their relevance. It brings deeper interaction between the mention and the type to reach better performance but has to perform $N$ (the type set size) forward passes to infer all the types of a single mention. CE is therefore very slow in inference when the type set is large (e.g., $N=10k$ for UFET). \% Cross-encoder also ignores the correlation between different types. To this end, we propose to perform entity typing in a recall-expand-filter manner. The recall and expansion stages prune the large type set and generate $K$ (typically much smaller than $N$) most relevant type candidates for each mention. At the filter stage, we use a novel model called {\{pasted macro `NAME'\}} to concurrently encode and score all these $K$ candidates in only one forward pass to obtain the final type prediction. We investigate different model options for each stage and conduct extensive experiments to compare each option, experiments show that our method reaches SOTA performance on UFET and is thousands of times faster than the CE-based architecture. We also found our method is very effective in fine-grained (130 types) and coarse-grained (9 types) entity typing. Our code is available at \{pasted macro `CODE'\}.