On a separate project I have, https://katchup.news/, I had been trying to make use of CloudFlare for my CDN and caching needs. CloudFlare is easy enough to setup and in general a really good tool. After I was done with the setup, I noticed a speed improvement and all was well.
That was until I noticed my backend services were not working properly. The site is updated through a backend service which interacts with an API at katchup.news. The error I was getting the following:
[Fatal Error] :1:1: Premature end of file.
redstone.xmlrpc.XmlRpcException: The response could not be parsed
And with this sprinkled about
lineNumber: 1; columnNumber: 1; Premature end of file. at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1239)
I knew the problem was CloudFlare-related because if I disabled the service, my backend services worked fine. One major problem I had with this error was I could not access the logs on CloudFlare (those seems to be part of an upgraded service), so I could not see what CloudFlare was getting. I also could not get much more from my backend logs. I was using a library and that’s all it was exposing.
With that in mind, I suspected the issue was around CloudFlare’s additional hop, its caching rules, and/or its security settings. Initially I thought there was nothing I could do about the hop, so I dug into the latter two. For the latter two I created a new page rule which excluded all of the caching and security features from the API end-point. I excluded everything. However, the issue remained.
The next thing I did was to create a firewall rule to achieve the same, but this also didn’t work.
Finally, after much frustration, I noticed that in the CloudFlare DNS settings, some entries were gray and others orange (the CloudFlare color) and that gray meant only DNS was passing through, but orange meant DNS and the CDN were set. Furthermore, if I enabled CloudFlare, but set the main DNS entry to gray (only DNS pass through), my backed service worked fine (which I think it’s the same as enabling/disabling at the main control panel).
That’s when I realized I could create a separate A record entry to use as a proxy for the API endpoint that CloudFlare’s additional hop was screwing up. I re-enabled CloudFlare in the standard way (set it to orange) for the main A record (katchup.news), created a new A record, api.katchup.news, without CDN, and created a subdomain (api.katchup.news) at my hosting service point to katchup.news. In this way I was able to enable all of CloudFlare’s features on the main domain (katchup.news), while still being able to access the API endpoint through the newly created A record.
With the above in place, the system is working as expected. Users going through katchup.news directly get the benefits of CloudFlare, while my backend services can still get undisturbed access the katchup.news APIs.