- Mozilla

This Week in Glean: Publishing Glean.js or How I configured an npm package that has multiple entry points

Wed, 07 Apr 2021 17:00:00 +0000

(“This Week in Glean” is a series of blog posts that the Glean Team at Mozilla is using to try to communicate better about our work. They could be release notes, documentation, hopes, dreams, or whatever: so long as it is inspired by Glean. You can find an index of all TWiG posts online).

All “This Week in Glean” blog posts are listed in the TWiG index (and on the Mozilla Data blog). This article is cross-posted on the Mozilla Data blog.

A few weeks ago, it came the time for us to publish the first version of Glean.js in npm. (Yes, it has been published. Go take a look). In order to publish a package on npm, it is important to define the package entry points in the project’s package.json file. The entry point is the path to the file that should be loaded when users import a package through import Package from "package-name" or const Package = require("package-name").

My knowledge in this area went as far as “Hm, I think that main field in the package.json is where we define the entry point, right?”. Yes, I was right about that, but it turns out that was not enough for Glean.js.

The case of Glean.js

Glean.js is an implementation of Glean for Javascript environments. “Javascript environments” can mean multiple things: Node.js servers, Electron apps, websites, webextensions… The list goes on. To complicate things, Glean.js needs to access a bunch of platform specific APIs such as client side storage. We designed Glean.js in such a way that platform specific code is abstracted away under the Platform module, but when users import Glean.js all of this should be opaque.

So, we decided to provide a different package entry point per environment. This way, users can import the correct Glean for their environments and not care about internal architecture details e.g. import Glean from "glean/webext" imports the version of Glean that uses the web extensions implementaion of the Platform module.

The main field I mentioned above works when the package has one single entry point. What do you do when the package has multiple entry points?

The `exports` field

Lucky for us, starting from Node v12.7.0, Node recognizes the exports field in the package.json. This field accepts objects, so you can define mappings for all your package entry points.

{
  "name": "glean",
  ...
  "exports": {
    "./webext": "path/to/entry/point/webext.js",
    "./node": "path/to/entry/point/node.js",
  }
}

Another nice thing about the exports field, is that it denies access to any other entry point that is not defined in the exports map. Users can’t just import any file in your package anymore. Neat.

We must also define entry points for the type declarations of our package. Type declarations are necessary for users attempting to import the package in Typescript code. Glean.js is in Typescript, so it is easy enough for us to generate the type definitions, but we hit a wall when want to expose the generated definitions. From the “Publishing” page on Typecript’s documentation, this is the example provided:

{
  "name": "awesome",
  "author": "Vandelay Industries",
  "version": "1.0.0",
  "main": "./lib/main.js",
  "types": "./lib/main.d.ts"
}

Notice the types property. It works just like the main property. It does not accept an object, only a single entry point. And here we go again, what do you do when the package has multiple entry points?

The `typesVersions` workaround

This time I won’t say “Lucky for us Typescript has this other property starting from version…”. Turns out Typescript, as I am writing this blog post, doesn’t yet provide a way for packages to define multiple entry points for their types declarations.

Typescript lets packages define different types declarations per Typescript version, through the typesVersions property. This property does accept mappings of entry points to files. Smart people on the internet figured out, that we can use this property to define different types declarations for each of our package entry points. For more discussion on the topic, follow issue #33079.

Back to our previous example, type definitions mappings would look like this in our package.json:

{
  "name": "glean",
  ...
  "exports": {
    "./webext": "path/to/entry/point/webext.js",
    "./node": "path/to/entry/point/node.js",
  },
  "typesVersions": {
    "*": {
      "./webext": [ "path/to/types/definitions/webext.d.ts" ],
      "./node": [ "path/to/types/definitions/node.d.ts" ],
    }
  }
}

Alright, this is great. So we are done, right? Not yet.

Conditional exports

Our users can finally import our package in Javascript and Typescript and they have well defined entry points to choose from depending on the platform they are building for.

If they are building for Node.js though, they still might encounter issues. The default module system used by Node.js is commonjs. This is the one where we import packages by using the const Package = require("package") syntax and export modules by using the module.exports = Package syntax.

Newer versions of Node, also support the ECMAScript module system , also known as ESM. This is the offical Javascript module system and is the one where we import packages by using the import Package from "package" syntax and export modules by using the export default Package syntax.

Packages can provide different builds using each module system. In the exports field, Node.js allows packages to define different export paths to be imported depending on the module system a user is relying on. This feature is called “conditional exports”.

Assuming you have gone through all the setup involved in building a hybrid NPM module for both ESM and CommonJS (to learn more about how to do that, refer to this great blog post), this is how our example can be changed to use conditional exports:

{
  "name": "glean",
  ...
  "exports": {
    "./webext": "path/to/entry/point/webext.js",
    "./node": {
      "import": "path/to/entry/point/node.js",
      "require": "path/to/entry/point/node.cjs",
    },
    ...
  },
  "typesVersions": {
    "*": {
      "./webext": [ "path/to/types/definitions/webext.d.ts" ],
      "./node": [ "path/to/types/definitions/node.d.ts" ],
      ...
    }
  }
}

The same change is not necessary for the ./webext entry point, because users building for browsers will need to use bundlers such as Webpack and Rollup, which have their own implementation of import/require statement resolutions and are able to import both ESM and CommonJS modules either out-of-the-box or through plugins.

Note that there is also no need to change the typesVersions value for ./node after this change.

Final considerations

Although the steps in this post look straightforward enough, it took me quite a while to figure out the correct way to configure the Glean.js’ entry points. I encountered many caveats along the way, such as the typesVersions workaround I mentioned above, but also:

In order to support ES6 modules, it is necessary to include the filename and extension in all internal package import statements. CommonJS infers the extension and the filename when it is not provided, but ES6 doesn’t. This get’s extra weird in Glean.js’ codebase, because Glean.js is in Typescript and all our import statements still have the .js extension. See more discussion about this on this issue and our commit with this change.
Webpack, below version 5, does not have support for the exports field and is not able to import a package that defined entry points only using this feature. See the Webpack 5 release notes.
Other exports conditions such as browser, production or development are mentioned in the Node.js documentation, but are ultimately ignored by Node.js. They are used by bundlers such as Webpack and Rollup. The Webpack documentation has a comprehensive list of all the conditions you can possibly include in that list, which bundler supports each, and whether Node.js supports it too.

Hope this guide is helpful to other people on the internet. Bye! 👋

This Week in Glean: Glossary

Fri, 10 Jan 2020 12:30:00 +0000

Last blog post: This Week in Glean: Migrations by me.

I am the newest Glean team member, having joined October/2019. One of the most challenging parts of understanding our product and our code base these past months has been keeping up with terminology. Coming from a web development background, some of the terms used in Glean have slight or big differences in meaning from that context. This has been an exercise in resignifying or flat out learning new words from the English language.

This blog post will be a sort of glossary with explanations and background on the terms that stood out to me.

The glossary

“Glean”, not “GLEAN” or “glean”

I think it is appropriate to start with the name of the product. Contrary to what I thought before joining the team, “Glean” is not an acronym. This may be obvious to those who already know the word, but I didn’t and the logo can be misleading since all the letters are capitalized.

The Glean logo. To know more about it, see This Week in Glean: What is the Glean logo about?

According to the dictionary the word “glean” means:

to gather information or material bit by bit

There are other meanings to it, of course. The one I quoted is the one that makes the most sense given what Glean is about.

I should also note that Glean is not a synonym to The Glean SDK, which is part, but not the whole of the Glean product. For more on “what is Glean exactly”, refer to the documentation.

Finally, as suggested by the title of this section, the “official” way to write “Glean”, is by capitalizing the word. Not all caps, not non-capitalized.

“metric”

Metrics are the individual things being measured using Glean. They are defined in metrics.yaml files. Glean itself provides some metrics out of the box and looking at its own metrics.yaml we can see some examples of what a metric can be:

The metric glean.baseline.duration of type timespan holds “The duration of the last foreground session.”
The metric glean.internal.metrics.os of type string holds “The name of the operating system.”
The metric glean.internal.metrics.first_run_date of type datetime holds “The date of the first run of the application.”

In previous versions of telemetry the word “probe” was used to describe a metric. The Glean team decided not to use that word anymore because it can be misleading since its meaning is not accurate to what a metric is.

For more information or questions on how to define or set metrics, refer to The Glean SDK Book.

“ping”

In the Glean context, a ping is an entity used to bundle related metrics. Out of the box the Glean SDK provides four default pings: the baseline ping, the metrics ping, the events ping, and the deletion-request ping. The Glean SDK also provides a way for the user to define their own custom pings.

For more information on custom or default pings, refer to The Glean SDK Book.

“submission”

In the Glean context, “to submit” means the to collect + to upload a ping.

The Glean SDK stores locally all the metrics set by it or by its clients. Each ping has its own schedule to gather all its locally saved metrics and create a JSON payload with them (e.g. every time the app goes to background for the baseline ping, or everyday at 4am for the metrics ping). This is called “collection”.

Upon successful collection, the payload is queued for upload, which may not happen immediately or at all (in case network connectivity is not available).

Unless the user has defined their own custom pings, they don’t need to worry too much about submitting pings. All the default pings have their scheduling and submission handled by the SDK.

For information on how to submit pings, refer to The Glean SDK Book.

Conclusion

Naming is hard and consistency is key.

This Week in Glean: Migrations

Fri, 06 Dec 2019 16:30:00 +0000

Last week’s blog post: This Week in Glean: Differences by jan-erik.

The Glean team has spent a bunch of 2019 rewriting the Glean SDK (which was initially written in Kotlin) in Rust and by the end of October we finally released that rewrite into the world. For more on that, refer to the previous TWiG blog post by jan-erik: A Release.

Of course, as with any major rewrite, we had to put some mechanisms in place so that a client that needed to transition from a previous version did so without any major problems. For Glean, that meant data migrations. Each metric type was locally stored a bit differently in the Kotlin implementation than it is now in the Rust one, so that had to be changed. The migration code was to be run the first time the updated Glean was initialized and only once.

To be sure that our users were migrating as expected, telemetry was added to know everytime a user migrated successfully.

Now that the Rust version had been released, it was time for us to check back on that and see if our users were migrating correctly. For that, I created two queries, one to verify that:

Only users who should have migrated (e.g. sent pings in previous versions), recorded successful migrations;
Users have recorded the successful migration only once;
Users have only recorded successful migrations from versions above 19 – which is the minimum version where they should.

And another one to verify that:

For the daily clients that should have migrated, they all have done so.

The first query gave us positive results, with no or very little clients showing any of the problems we were looking for. Great!

The second query, also gave positive results with ~97,5% of our daily clients having recorded succesful migrations. Unfortunately, those ~2,5% that did not record successful migrations were a concern that needed to be verified.

Off I went to write yet more SQL, the question I wanted to answer this time being:

Are the daily users that have not migrated always the same ones? Or are we getting different users everyday?

Since I didn’t know how to answer exactly that with only SQL, I changed the question to:

How many metrics pings does a user send after updating, but before recording the successful migration?

I found that for the users that have recorded successful migrations, usually that count ranges between 0-2 pings, but for the users that haven’t recorded the successful migration the count is between 1-73 pings.

(It should be noted, that in a perfect world, all users should send 0 pings after updating, but before migration. Migration should happen immediatelly and all pings should happen after that, but that is not happening due to another issue.)

Back to the users that haven’t recorded a successful migration: the fact that they have recorded so many pings without recording the migration are already an indication that they have successfully migrated and for some reason haven’t added that to the ping. Because of the way the migration was implemented, if the user doesn’t successfully migrate, they will get a new client id, thus making it impossible for us to relate their pings from the new version with their pings from previous versions.

And that is that! I have still to find out why some clients are not correctly recording the successful migration and if you are curious about that, check on this bug. I might have found that out by the time you read this blog post.

End of Outreachy

Sat, 16 Sep 2017 17:12:00 +0000

So, Outreachy is officially over and I can’t express how much I loved it. Mike was the greatest mentor that I could ask for, super patient and helpful, but at the same let me do what I really wanted to. I will continue working with the Webcompat team, because I really liked the problems and also because there is still a lot of stuff to do regarding functional and unit tests.

I was lucky enough to get an internship that will start on the day after Outreachy ends, so I won’t be unemployed for any time at all.

The end of Outreachy also doesn’t mean the end of this blog. I really enjoyed sharing my experiences here, it helps me think and review a lot of stuff. I’ll try and continue writing everytime that I can or when I get something really cool to share.

That’s it for now. ‘Till next time.

BrazilJS Conference

Fri, 01 Sep 2017 17:12:00 +0000

Outreachy gives us a grant of up top $500 for traveling, be it for a conference, a workshop, a course, anything that has something to do with the Open Source Community. My mentor, Mike Taylor, was going to come to BrazilJS and asked me if I wanted to use my travel allowance for it, I agreed and went to the conference that happened in Porto Alegre.

BrazilJS

BrazilJS is self proclaimed as the “biggest Javascript conference in the universe” and roughly 1.500 people attend it every year in Brazil. My mentor, Mike Taylor, lived in Brazil for sometime in his life and knows how to speak Portuguese really well. He has been attending the conference for the past years and this year was no exception. Since I live in Sao Paulo, Outreachy gives a travel allowance, and in my work I use a lot of Javascript, it made a lot of sense for me to go with, and I did.

Mike was going to give a talk there about “weird APIs” and Mozilla itself was going to have a booth about WebVR and A-Frame, so I went as a Mozilla employee.

The conferece went on for two days and I watched a lot of great talks. The ones that I enjoyed the most were the WebVR ones “3D Experiences + The Web Platform” and “A-Frame 101”. The first one was mostly just a presentation of stuff that you can do with three.js, A-Frame and Web Assembly and the second was an introduction and showcase of A-Frame and it’s features.

A-Frame

I have to make a section only for A-Frame in this blog post, because it is too awesome.

As I understand it, A-Frame is a wrapper around the three.js framework and makes it super really easy to build virtual reality experiences with just HTML and JavaScript. The feature that left me awed was the inspector one. Basically, when you have an A-Frame scene in a webpage, you can add the inspector feature to it just my injecting some JavaScript to the page. When you open it, you can press Alt+Shift+i and instead of the browser inspector opening up, the A-Frame inspector will.

The A-Frame inspector is a simple yet powerful 3D modeling tool for the browser. You can add, move, change, update, animate and do a lot of cool stuff to 3D models and scenes directly from the browser.

I will add links to all this in the related links section for this post.

Conclusion

BrazilJS was my first ever conference. If I’m being really honest, I have a really hard time just staying put and watching talks, but I really don’t think that a conference is all about that. Of course the talks are really important, but what I brought home with me that I treasure more than the talks that I watched were the people that I met, the stories that I listened to, the experiences that I shared with everyone that I met there and of course the t-shirts that I got.

Promises and async / await with Intern.js

Sun, 20 Aug 2017 17:12:00 +0000

As I mentioned in my previous blog post about Browserstack, I got a lot of errors when I tried to run our Intern.js functional tests in browsers other than Chrome and Firefox. I left this issue aside at the time, because it was too saturated in my mind, but I decided to give it another go these past two weeks.

Understanding how Intern.js works

The framework that we use for Javascript functional testing is Intern.js. Intern.js has a tool called Leadfoot, that puts a layer of abstraction over the Selenium WebDriver API and makes it cross browser compatible.

Here is an example of a simple Intern.js test that we have in the Webcompat tests:

"Comments form visible when logged in": function() {
    return (
      FunctionalHelpers.openPage(this, url("/issues/100"), ".js-Issue")
        // Comment form visible for logged in users.
        .findDisplayedByCssSelector(".js-Comment-form")
        .end()
    );
  },

Each one of the functions that is called inside of the tests return a promise.

Debugging Safari tests

The first thing I did to start debugging the Safari tests was modify the Intern.js config file to have the tests run on Safari. After I ran them a bunch of times, I realized that most of the errors I was getting were actually intermittent (they happened randomly). I fixed a lot of other intermittent errors in the tests before, and all of them where latency problems. Some function would return after or before expected.

Since all of the Intern.js/Leadfoot functions return Promises, I thought that maybe it was a problem of a function that resolved after expected and broke the code. So, I took one single test that failed in Safari continuously and decided to use it as a proof of concept.

The test before I changed anything looked like this:

"Add a screenshot to a comment": function() {
    return FunctionalHelpers.openPage(
      this,
      url("/issues/100"),
      ".wc-Comment-body"
    )
      .findById("image")
      .type("tests/fixtures/green_square.png")
      .end()
      .sleep(2000)
      .findByCssSelector(".js-Comment-text")
      .getProperty("value")
      .then(function(val) {
        assert.include(
          val,
          "[![Screenshot Description](http://localhost:5000/uploads/",
          "The image was correctly uploaded and its URL was copied to the comment text."
        );
      })
      .end();
  },

I tried to modify it to call the Promises in a synchronous manner, which means, only call the next function once the previous one has resolved. The code looked horrible, but it worked, the test started passing. Here is what it looked like:

"Add a screenshot to a comment": function() {
	var remote = FunctionalHelpers.openPage(
	  this,
	  url("/issues/100"),
	  ".wc-Comment-body",
	    true
	  );
	  return (
	    remote
	      // Find image upload input.
	      .findById("image")
	      .then(function(el) {
	        el
	          // Type fixture image path into it.
	          .type(require.toUrl("../fixtures/green_square.png"))
	          .then(function() {
	            remote
	              .end()
	              // Find the comment textarea.
	              .findByCssSelector(".js-Comment-text")
	              .then(function(el) {
	                el
	                  // Get it's value.
	                  .getProperty("value")
	                  .then(function(val) {
	                    // Check that the value is what expected.
	                    assert.include(
	                      val,
	                      "[![Screenshot Description](http://localhost:5000/uploads/",
	                      "The image was correctly uploaded and its URL was copied to the comment text."
	                    );
	                  });
	              });
	          });
	      })
	      .end()
	  );
},

At the moment we have a total of 76 functional tests in 17 different files. It would be very difficult to maintain code like this. It’s too extensive and difficult to read. So I started researching some other way to write it and make it look good and be easy to maintain, that’s when I stumbled upon async / await.

Understanding async / await

When you have code that is heavy with Promises, async / await is a great tool to help you organize and synchronize your code. I think the best way to understand it is with a simple example:

function simplePromise() {
    return new Promise(function(resolve) {
        setTimeout(resolve, 10000);
    });
}

async function exampleAsync() {
	await simplePromise();
	console.log('Promise is done!');
}

If you run this example in your console, you will see that ‘Promise is done!’ will only be printed after the time interval. That is what async / await does. It holds of code execution until a promise is returned. The usage is pretty simple. It’s important to note that for you to be able to use await you have to be inside of an asynchronous function and for a function to asynchronous you need to add the keyword async before declaring it.

Using async / await in our tests

I used the same test and applied the async / await idea to it. It looked like this:

"Add a screenshot to a comment": async function() {
    await FunctionalHelpers.openPage(
      this,
      url("/issues/100"),
      ".wc-Comment-body"
    );

    let input = await this.remote.findById("image");
    await input.type(require.toUrl("../fixtures/green_square.png"));

    let textarea = await this.remote.findByCssSelector(".js-Comment-text");
    let text = await textarea.getProperty("value");

    assert.include(
      text,
      "[![Screenshot Description](http://localhost:5000/uploads/",
      "The image was correctly uploaded and its URL was copied to the comment text."
    );
},

This looks much better than the first time I tried the idea. Interestingly enough, it didn’t work.

Unfortunately, this blog post will end in a sad note as I didn’t find out the solution for this and Safari tests are still on hold. We had some other more necessary features to add to the tests and my internship is almost over, so we left it aside, once again.

If anyone knows the solution for this, please write me or open an issue in the repository for this blog.

Travis CI and the Submission of PR's

Sat, 05 Aug 2017 17:12:00 +0000

One of the goals for my internship was to get the tests of pull requests submitted from forks to work, because they were not working and the reason was unknown. Here I will explain how I did it and how simple it was.

What is TravisCI

TravisCI is a continuous integration tool for GitHub test driven development. How it works is really simple, it provides us access to a remote virtual machine that we can control with a GUI and with a .travis.yml file. We use this machine to run a series of commands in the branch that is being sent by the a newly submitted pull request or commit.

To make things clearer, I’ll just explain how we use it in Webcompat.

Everytime that a user sends in a commit or a pull request, we have a setup Travis configuration file that downloads the new code, builds the webcompat environment for development and runs the unit and functional tests to check that everything is alright. If all tests are passing, the pull request will have a small green tick after it’s title, if it doesnt’t the pull request will have a small red ‘x’ after it’s title. We can also manually re-run all tests in a pull request as many times as we want.

What was the problem with our failing builds

In the Webcompat repository, we had a really annoying problem that was, even if all tests were passing in a pull request, if it came from a fork they would fail. That was even more annoying given that even some of our core contributors use forks instead of just using the webcompat repository.

Luckly the problem was really simple.

Travis has some “secure variables” that are only available to the owners of the main repository being tested. Our tests were relying in some of these secure variables, because we had to add GitHub token and secret in order to access the GitHub API for running tests. Now, the only thing I had to do was remove all these secure variables, because, as I explained in my previous blog posts, one of the things that I did to the Webcompat unit and functional tests, was mock all GitHub API calls. Everything worked.

Conclusion

Sometimes the solution for a coding problem can be super simple. When I first started to tackle this problem I was trying all this crazy workarounds and I actually didn’t even need them.

Mocking GET requests with Python unittest.Mock

Thu, 20 Jul 2017 17:12:00 +0000

At the Mozilla All Hands, I finished getting all functional tests to work offline. Which means that I got everything to return fixture data and none of our API calls to ever go to an external website. But while in the All Hands I also realized that some tests from our Python unit tests were network dependant and needed to be mocked.

unittest.Mock

Our unit tests are written in Python, because our application is in Flask. So, to make them network independent I had to mock everything out in Python. That’s when I realized it was time to study and understand the Python unittest.Mock class. Because it provides us with a series of helper classes and methods to mock any function or object that you may need in a Python test.

In my case, I only needed to mock GET requests to the GitHub API. I had to patch the API call and return the expected data. In order to do that, understanding only one concept was key for me to know what I had to do: I only needed to patch the API call to GitHub, all the logic in the Webcompat application had to stay intact, or else I would be left with testing blind spots.

First I familiarized myself with the exact architecture of the webcompat application, to know which function I had to patch in order to only mock what was necessary. After doing that, it was very stright forward: patch the function and return the expected data from it.

Here is an example of one of the functions that I patched. Mind you, the function that made the API call is the webcompat.helpers.proxy_request:

def test_api_issues_out_of_range(self):
    '''API issue for a non existent number returns JSON 404.'''
    with patch('webcompat.helpers.proxy_request') as github_data:
        github_data.return_value = mock_api_response({
            'status_code': 404,
            'content': '[{"message":"Not Found","documentation_url":"https://developer.github.com/v3"}]'  # nopep8
        })
        rv = self.app.get('/api/issues/1', environ_base=headers)
        json_body = json.loads(rv.data)
        self.assertEqual(rv.status_code, 404)
        self.assertEqual(rv.content_type, 'application/json')
        self.assertEqual(json_body['status'], 404)

You can see that I don’t even need to return all the response that the GitHub API would, only the parts that I would need for everything to work right.

All Hands 2017 San Francisco

Wed, 05 Jul 2017 23:56:00 +0000

Mozilla has a biannual gathering for all employees and some volunteers, it’s called the Mozilla All Hands and it always happens in a different city around the world. This year, the Summer All Hands happened in San Francsico, CA, USA and as an intern I had the priviledge to go and experience the event.

About the All Hands and San Francisco

I had never gone to something like this, I had never gone to a programming conference, I hadn’t even ever gone to the United States. Needles to say, I was super excited.

They let me know about the conference about a month before, for me to have time to get the plane tickets get a hotel room and prepare psychologically for the trip. Mozilla would pay for everything, so it was not much of a thinker, of course I accepted the invitation and went.

The All Hands is a work week. Since many Mozilla employees work remotely or even the ones that work in offices usually have a team that is scattered aroud the globe, it’s important to have this sort of event where everybody gets together, meets, works and has fun.

I arrived there around 1PM a day before the first day of the meeting. I was super excited to have a free day to walk around and get to see a little bit of San Francisco before getting to work. I was really surprised as to how the United States reminded me of Brazil. Having just come back from a three month internship in Germany, where there is a reallly big difference from Sao Paulo, I was surprised that San Francisco was not much a break from Brazil. Of course, the city is much smaller and there is the different language, but other than that, the vibe doesn’t change much.

Even though it was so much like Brazil, the city was new to me so I wanted to walk around. I saw the cable car, saw the bay, went into some stores and just tried to experience as much as I could just by walking around the hotel. Luckly the hotel was in a really central area so I could see a lot. At the end of the day I was really tired, but satisfied and so I went to the welcome cocktail to the All Hands.

At the cocktail I got to finally meet Mike in person and some other Mozilla employees. I didn’t stay much, because the tiredness from the almost 15 hour trip and the jetlag was starting to get to me. I decided to go to sleep so as to fully enjoy the next day.

The actual work week

The first day started out with a really brief talk to get things going and explain how the week would go about. We were told that the week would not be about going to meetings and going to talks, it would be about sitting down and coding a lot with your colleagues. And that’s what we did. For the next five days I would get up in the morning, have breakfast go to the office that was setup for us and work a lot.

Even though I coded so much, I don’t think that that was the best part of the week, by far. It was really great for me to get to know all the other guys from the Webcompat team and the other girls that were also Outreachy interns.

It was great to hear the stories of all my other team colleagues and learn with them. I also really enjoyed getting to know what they did for the project. Some of the guys worked triaging and finding the solution for the compatibility bugs that are filed on webcompat.com, others worked improving the design of the website, others the functionality and so on, some guys were even working on browser tools to help debug the compatibilioty bugs that we get. Awesome.

Imposter syndrome workshop

Aside from the coding, Mozilla provided a workshop only for the Outreachy interns about Imposter Syndrome and how to try and overcome it.

Imposter Syndrome is a really common evil and specially present among girls that work in tech. Basically, it’s the feeling that you are not good enough, but so intense that it becomes almost pathological. It was great to hear the complaints, fears and ways to improve that the other girls shared during the workshop. Each one had a different story and by hearing and identifying myself with them I felt much better about me and my work. It made me feel part of something and not so alone in my struggles and fears.

Conclusion

I thought the All Hands was a key event for me to feel as if I was a part of the Mozilla team. Even though I saw the avatars and usernames of all my colleagues online, there is nothing better than meeting everyone in person. One big thing that changed for me is my online presence with the team. Now that I know everybody in person I get much less shy about talking in IRC and GitHub and in video confereces with the group.

It was also great to get the time to have fun with them, not only work. I got to go to a record store with Mike (I’m a vynil collector as is he), got to eat some burritos with the rest of the team and also go to a fancy restaurant and we even did some sightseeing. All in all it was a great week.

Flickr Group with a lot of pictures from the meeting

Cross Browser Testing with Browserstack

Wed, 21 Jun 2017 17:12:00 +0000

This will be another Outreachy diary.

One of my goals for the internship was to user Browserstack for cross browser testing. That is important, because right now we are only testing in Firefox, but our users can come from any browser, so we should test with as many as possible.

Trying Browserstack out

Browserstack a “free for open source projects” tool, that let’s you use and run automated tests in remote machines with different operating systems, different browsers and even on mobile devices. Our functional tests are written using Intern.js, an awesome functional testing tool written in JavaScript, that uses Selenium to automate tests in any browser. Other than that, Intern.js has Browserstack Tunnel built in, so that the integration is painless.

Even though Browserstack is a great tool, I ran into quite a lot of problems while trying to do cross browser testing with it. I’m going to list them all here.

Uploading local files

One of the functionalitites that we check in our Intern.js tests is file uploading. The problem that I ran into was that, even though the tests were running in the Browserstack remote machine, the path for the test file that we wanted to upload was referencing my local machine and that returned a “File not found” error.

I looked for quite a while into that, but nothing seemed to work. It was a really weird error, because when I looked into Leadfoot’s documentation it explained that when you tried to upload a file through a file input the driver would seamlesly upload the file to the remote server before uploading it through the input.

Anyways, I was completely lost on this issue, so I asked a question in Stack Overflow (my first question there ever) and a guy really helped me out. The only thing I needed to do was add this to the Intern.js configuration object:

capabilities: {
    fixSessionCapabilities: true,
    remoteFiles: true,
    .
    .
    #.
  },

Latency

Now that I got the uploading out of the way, I could just start testing in different browsers. That was when I ran into another issue. It’s great to test everything in remote machines, but it has a real set back: it is really slow. Because of the latency, many tests would time out or not run at all. For that I found no solution.

Inconsi#stencies across browsers

Since we only tested with Firefox, we didn’t have a real idea of how the tests would go about in diferent browsers. For starters, I tried testing with Chrome and that went well. The results were not so different from Firefox, one minor change to the tests code and everything passed.

Now it was time to try out Edge. When I started the internship, Mozilla sent me a MacBook Pro, so I didn’t even have Edge to test with locally, anyways. The results were sad. Many of the tests didn’t pass, the error messages were not really clear and I couldn’t even test locally to have a better and faster development experience. I explained the issue to Mike and after checking Google Analytics and realizing that less than one percent of our users came from Edge (really should have looked at that first thing right, you live and learn) we decided to leave it aside and go back to it when we have time or if ever it becomes really important.

Finally it was time to test with Safari. Those were the most depressing results of all. The majority of tests would fail in Safari remote and locally, nothing made much sense and hope was crushed. Safari is the third most used browser among our users, but still it’s not even 5% of our user base, so we decided to also leave it aside.

One import observation about this section is that even though tests were not passing, it didn’t mean that the stuff being tested was not working. I tried testing manually and proved that. The problem was with Selenium or Leadfoot, so it was not crucial that we fixed it rig#ht away.

Conclusion

Phew.

After all this, I think I don’t even have to say that I was frustrated with Browserstack. And then I got to thinking: Do we really need to use it?

I added Chrome testing to our TravisCI Pull Request automated testing code and I found out that I could use OS X in Travis as well, so that I could test Safari with it too. And that was my answer: I didn’t really need to use it.

I presented to Mike my findings and he agreed that it was not necessary and we closed all issues about it.

I think Browserstack is a really good tool if your application has users coming from all sorts of places from mobile devices to diferent operating systems and different browsers, but it turns out that is not the case for Webcompat. Our users come from Desktop devices almost exclusively using Firefox or Chrome and Browserstack was overkill and an unnecessary burden.

Now what is left to be done is see what is going on with Safari and get all tests running with it so that we can also test there, but I think I should leave this issue aside for now since it’s too saturated in my mind at the mo#ment.

- Mozilla

This Week in Glean: Publishing Glean.js or How I configured an npm package that has multiple entry points

The case of Glean.js

The exports field

The typesVersions workaround

Conditional exports

Final considerations

This Week in Glean: Glossary

The glossary

“Glean”, not “GLEAN” or “glean”

“metric”

“ping”

“submission”

Conclusion

This Week in Glean: Migrations

End of Outreachy

BrazilJS Conference

BrazilJS

A-Frame

Conclusion

Related Links

Promises and async / await with Intern.js

Understanding how Intern.js works

Debugging Safari tests

Understanding async / await

Using async / await in our tests

Related Links

Travis CI and the Submission of PR's

What is TravisCI

What was the problem with our failing builds

Conclusion

Related links

Mocking GET requests with Python unittest.Mock

unittest.Mock

Related Links

All Hands 2017 San Francisco

About the All Hands and San Francisco

The actual work week

Imposter syndrome workshop

Conclusion

Related Links

Cross Browser Testing with Browserstack

Trying Browserstack out

Uploading local files

Latency

Inconsi#stencies across browsers

Conclusion

Related Links

The `exports` field

The `typesVersions` workaround