Excessive disk writes by ABP

Posting here is no longer possible, please use the forum of a filter list project, such as EasyList
Locked
UnregisteredUser101

Excessive disk writes by ABP

Post by UnregisteredUser101 »

Good evening and thank you for making ABP.

Now to the problem. A few days after adding ABP extension to FF, I noticed that FF had written 40 GB in just 4 days, based on the "IO Write Bytes" value in Task Manager. That's 10 GB per day.

During that time period, I have visited several web sites but NO streaming video and NO downloading of big files.

After briefly monitoring FF with Sysinternals' Process Monitor, I noticed that FF made hundreds of writes in just 1 minute to the file "patterns.ini.tmp" and, during that time, the "IO Write Bytes" value increased by 500 MB.

I don't see any file named "patterns.ini.tmp" in my FF profile. I'm guessing that it gets renamed to "patterns.ini".

I have these 4 filter subscriptions: EasyList, MalwareDomains, Fanboy and EasyPrivacy.

The total size of the adblockplus folder in my FF profile is around 18 MB. Even if the filters are updated more than once a day, something doesn't add up...

Or maybe the "IO Write Bytes" value isn't what I think it is. AFAIK, it also accounts for data sent, not received, which should be insignificant.

Please enlighten me or tell me it's a bug and I'll give more details.

I have FF 29.0.1 and ABP 2.6 and a lot of RAM.

TIA.
UnregisteredUser101

Re: Excessive disk writes by ABP

Post by UnregisteredUser101 »

I might have found a way to reproduce (I can't edit my post cause I'm not reg'ed).

The problem can be triggered by going to http://www.radio-canada.ca/tele/guide-horaire. You might have to wait for up to a minute after the page has loaded. Then it starts. Non-stop disk writes until I closed the tab. Then it stops.

This time over 1 GB was written before I closed the tab.

Looking at Process Monitor's output, it seems like maybe patterns.ini is updated not just once but many times in a loop?

I used my firewall to monitor active connections, and Firefox made a few connections but only a few kilobytes were transferred.

Not all web pages trigger the problem. For instance, visiting this forum is fine.


Win7 SP1 64-bit
Firefox 29.0.1
ABP 2.6 (NO custom filers)
NoScript 2.6.8.26
Other extensions probably not worth mentioning...
EMET 4.1
Comodo Internet Security Premium 7
kreemoweet
Posts: 20
Joined: Wed Jul 04, 2012 6:22 pm

Re: Excessive disk writes by ABP

Post by kreemoweet »

Confirmed on Windows Vista, ABP 2.6.3, FF 29.0.1. Not only does FF start writing incessantly (as described) to patterns.ini.tmp, but
CPU usage is pegged at 50-60% (normal would be 2-3%), a minute or two after loading the site mentioned above.
UnregisteredUser101

Re: Excessive disk writes by ABP

Post by UnregisteredUser101 »

Indeed, I forgot to mention it but you are right about the CPU usage.

(And I meant ABP 2.6.3, not just 2.6)
UnregisteredUser101

Re: Excessive disk writes by ABP

Post by UnregisteredUser101 »

Tested it again and this time I noticed something else. When loading the site above, the most active filter is ||2o7.net^$third-party and the hit count for that filter increases rapidly over time. It has over 250 hits after a minute or so. Then the excessive disk writing begins while the hit count continues to increase.

In total that filter has over 26,000 hits, by far the most active filter. This may be a clue.
User avatar
mapx
Posts: 21940
Joined: Thu Jan 06, 2011 2:01 pm

Re: Excessive disk writes by ABP

Post by mapx »

try this exception filter:

Code: Select all

@@||2o7.net/b/ss/$image,domain=ici.radio-canada.ca
User avatar
mapx
Posts: 21940
Joined: Thu Jan 06, 2011 2:01 pm

Re: Excessive disk writes by ABP

Post by mapx »

kreemoweet wrote:Confirmed on Windows Vista, ABP 2.6.3, FF 29.0.1. Not only does FF start writing incessantly (as described) to patterns.ini.tmp, but
CPU usage is pegged at 50-60% (normal would be 2-3%), a minute or two after loading the site mentioned above.
when/where are you noting this behaviour, on which page / site ? if radiocanada (and easyprivacy enabled) add the filter above
User avatar
mapx
Posts: 21940
Joined: Thu Jan 06, 2011 2:01 pm

Re: Excessive disk writes by ABP

Post by mapx »

all the filters (official enabled lists + custom filters) are in the patterns.ini file.
unfortunately if the option "count filter hits" is enabled, the statistics are re-written in the same patterns.ini file

in case of some troubled page (like radio-canada which wants that image 1x1 size from 2o7.net site and retry again and again) the filter 2o7.net will be triggered continuously and the statistics are written again and again

however in the development roadmap there are some improvements to do, like that to keep separated the statistics from filters:
https://issues.adblockplus.org/ticket/495
Moving filter hit statistics out of patterns.ini and into a separate file might be a good idea, the hit count is already responsible for much of the file size there.
UnregisteredUser101

Re: Excessive disk writes by ABP

Post by UnregisteredUser101 »

Thanks for the clear explanation, mapx. Indeed, your exception filter solves the problem for this site. But I chose to disable "count filter hits" since it could happen on other sites. You could even make it the default setting (disabled). Imho, most users probably don't care about stats as long as the ads are blocked.
MonztA
ABP Developer
Posts: 3957
Joined: Mon Aug 14, 2006 12:18 am
Location: Germany

Re: Excessive disk writes by ABP

Post by MonztA »

Locked