We have hosted the application robots io in order to run this application in our online workstations with Wine or directly.


Quick description about robots io:

Robots.io is a Java library designed to make parsing a websites 'robots.txt' file easy.

The RobotsParser class provides all the functionality to use robots.io.

Domains passed to RobotsParser are normalised to always end in a forward slash. Disallowed Paths returned will never begin with a forward slash. This is so that URL's can easily be constructed.

Programming Language: Java.

.

Page navigation:

©2024. Winfy. All Rights Reserved.

By OD Group OU – Registry code: 1609791 -VAT number: EE102345621.