Brushing aids in breaking up biofilms, algae, water mold (even if you may not see these problems).
Make sure the filter is cleaned often depending regarding how much are generally using the pool.