After some tests with the said document from https://github.com/koreader/koreader/issues/970 it seems like that the visual defect occurs for only semi-auto and manual crop modes. I've removed auto-crop from the rule so fixes https://github.com/koreader/koreader/issues/4106 when using auto-crop.