Abstract: Chain-of-thought (CoT) prompting, i.e., step-by-step reasoning instruction, enhances the performance of large language models (LLMs) in various tasks. Nevertheless, whether this gain suggests that LLMs have robust reasoning abilities remains unclear. In this study, we inspect the LLMs' step-by-step reasoning ability under controlled settings, with a particular focus on negation. Our results indicate that all tested LLMs were not robust against lexical negation (e.g., plausible -> implausible) when performing the CoT reasoning. Furthermore, our experiments with varying levels revealed that different LLMs exhibit different difficulties and output biases against lexical negation.