Display doorhanger sooner?

[pdehaan]

Per #37 (comment)

For co188.com, the popup isn't showing up because the page takes forever to finish loading. The popup should be displayed once the page loads.

I've seen this on other sites as well. If you have a site that takes forever to finish loading (i think the other one i was testing was last.fm), you don't see the alert unless you stay on the site/page long enough for the doorhanger to hang. Not sure if we should try displaying the doorhanger sooner so the user's have something amusing to read while they wait for the site to load.

[nhnt11]

Yeah, I'm not sure what the best way to fix this is. I'd have to leave it to UX to make the call. The popup was originally being shown immediately when the page STARTED loading, but based on Jacqueline's feedback I changed it to show AFTER the page is done loading. We didn't think about the case where it takes a long time for the page to load.

Ideally I think we should show the popup when the page being loaded has displayed some content - i.e. don't show it until the page is no longer blank. Might be a technical challenge though.

[pdehaan]

Hardly scientific, but I was having fun trying to get some timing data out of puppeteer. This only scrapes the first 50 breach URLs, but we can see that requests can take anywhere from 1.1 seconds up to 55+ seconds...

Not sure if our current telemetry/GA tells us when the doorhanger was displayed (relative to page loads).

const breaches = require("blurts-addon/src/breaches.json");
const puppeteer = require("puppeteer");

async function main(count=50, start=0) {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.setViewport({ width: 1280, height: 700 });

  for (const {Domain, Name} of breaches.splice(start, count)) {
    if (Domain) {
      await scanSite(page, Domain, Name)
        .catch(err => console.error(`ERROR! ${Domain}: ${err.message}`));
    } else {
      console.log(`WARNING! No domain found for ${Name}`);
    }
  }
  await browser.close();
}

async function scanSite(page, Domain, Name) {
  const timerLabel = `${Name}_page.load`;
  console.time(timerLabel);
  page.once('load', () => console.timeEnd(timerLabel));
  await page.goto(`http://${Domain}`);
}

main(50);

$ node index

000webhost_page.load: 4192.380ms
126_page.load: 5971.981ms
ERROR! 17app.co => net::ERR_NAME_NOT_RESOLVED at http://17app.co
17Media_page.load: 15260.781ms
17173_page.load: 15228.280ms
WARNING! No domain found for 2844Breaches
2fast4u_page.load: 3884.612ms
ERROR! 7k7k.com => Navigation Timeout Exceeded: 30000ms exceeded
7k7k_page.load: 41706.232ms
8tracks_page.load: 11703.454ms
Abandonia_page.load: 10615.393ms
AbuseWithUs_page.load: 1541.552ms
AcneOrg_page.load: 11033.321ms
Adobe_page.load: 8523.606ms
AdultFriendFinder_page.load: 6811.688ms
AhaShare_page.load: 2691.347ms
AIType_page.load: 8574.220ms
Aipai_page.load: 16656.002ms
ERROR! akparti.org.tr => Navigation Timeout Exceeded: 30000ms exceeded
AKP_page.load: 50298.290ms
Ancestry_page.load: 20294.707ms
AndroidForums_page.load: 16885.752ms
WARNING! No domain found for AntiPublic
ArmyForceOnline_page.load: 10017.957ms
AshleyMadison_page.load: 7683.816ms
ERROR! astropid.com => net::ERR_NAME_NOT_RESOLVED at http://astropid.com
AstroPID_page.load: 15730.444ms
Aternos_page.load: 15702.698ms
Autocentrum_page.load: 12805.790ms
Avast_page.load: 21861.449ms
WARNING! No domain found for B2BUSABusinesses
Badoo_page.load: 13116.144ms
ERROR! battlefieldheroes.com => Navigation Timeout Exceeded: 30000ms exceeded
BattlefieldHeroes_page.load: 55203.285ms
BeautifulPeople_page.load: 25199.784ms
Bell_page.load: 28952.279ms
ERROR! bell.ca => Navigation Timeout Exceeded: 30000ms exceeded
Bell2017_page.load: 5484.478ms
ERROR! bestialitysextaboo.com => Navigation Timeout Exceeded: 30000ms exceeded
Bestialitysextaboo_page.load: 31139.471ms
BigMoneyJobs_page.load: 1134.653ms
ERROR! binweevils.com => Navigation Timeout Exceeded: 30000ms exceeded
BinWeevils_page.load: 35143.225ms
BiohackMe_page.load: 5140.391ms
BTSec_page.load: 10930.916ms
BitcoinTalk_page.load: 10240.041ms
ERROR! bitly.com => Navigation Timeout Exceeded: 30000ms exceeded
ERROR! bittorrent.com => Navigation Timeout Exceeded: 30000ms exceeded
Bitly_page.load: 80649.498ms
BitTorrent_page.load: 50645.401ms
BlackHatWorld_page.load: 20644.995ms
ERROR! bolt.cd => net::ERR_NAME_NOT_RESOLVED at http://bolt.cd
ERROR! botoflegends.com => Navigation Timeout Exceeded: 30000ms exceeded
ERROR! forums.boxee.com => net::ERR_NAME_NOT_RESOLVED at http://forums.boxee.com
Bolt_page.load: 41442.028ms
BotOfLegends_page.load: 31411.015ms
Boxee_page.load: 1403.552ms
Brazzers_page.load: 11.111ms
BTCE_page.load: 25624.003ms
BusinessAcumen_page.load: 25615.757ms
CafeMom_page.load: 18360.537ms
CannabisForum_page.load: 8051.262ms
CashCrate_page.load: 21976.568ms

[pdehaan]

Actually, here's a version that uses puppeteer and the window.performance.timing API instead of console.time():

const breaches = require("blurts-addon/src/breaches.json");
const puppeteer = require("puppeteer");

async function main(count=50, start=0) {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.setViewport({ width: 1280, height: 700 });

  console.log("DOMAIN | PAGE_LOAD | NOTES\n-------|----------:|------");
  for (const {Domain, Name} of breaches.splice(start, count)) {
    if (Domain) {
      try {
        const res = await scanSite(page, Domain, Name)
        console.log(res);
      } catch (err) {
        console.error(`${Domain} | n/a | ${err.message}`);
      }
    } else {
      console.log(`${Name} | n/a | No domain found.`);
    }
  }
  await browser.close();
}

async function scanSite(page, Domain, Name) {
  await page.goto(`http://${Domain}`, {waitUntil: 'load'});

  const performance = JSON.parse(await page.evaluate(() => JSON.stringify(window.performance)));
  return `${Domain } | ${Math.round(performance.timing.loadEventEnd - performance.timeOrigin).toLocaleString()}ms`;
}

main(20);

Output:

DOMAIN	PAGE_LOAD	NOTES
000webhost.com	2,767ms
126.com	7,320ms
17app.co	n/a	net::ERR_NAME_NOT_RESOLVED at http://17app.co
17173.com	16,131ms
2844Breaches	n/a	No domain found.
2fast4u.be	3,252ms
7k7k.com	n/a	Navigation Timeout Exceeded: 30000ms exceeded
8tracks.com	5,500ms
abandonia.com	7,707ms
abusewith.us	1,346ms
acne.org	2,445ms
adobe.com	1,947ms
adultfriendfinder.com	2,160ms
ahashare.com	2,246ms
aitype.com	3,069ms
aipai.com	14,117ms
akparti.org.tr	6,726ms
ancestry.com	3,015ms
androidforums.com	3,876ms
AntiPublic	n/a	No domain found.

[tcinotto]

For MVP: timeout needed yes or no? Out of scope for MVP.

For sites that take a long time to load and a user navigates away from site, they will not see door hanger. Door hanger does not appear until the site has fully loaded.