Isopack Format

Technical documentation of Meteor’s Isopack package format

Isopack Format

An Isopack is Meteor’s package format containing compiled source code for each architecture the package supports. Each architecture-specific build is called a “Unibuild”.

Definition

From GLOSSARY.md:

Each package used by Isobuild forms an Isopack. Isopack is a package format containing source code for each architecture it can be ran on. Each separate part built for a separate architecture is called “Unibuild”.

Isopack Structure

An Isopack contains:

// From isopack.js
var Isopack = function () {
  // Package metadata
  self.name = null;
  self.metadata = {};
  self.version = null;
  self.isTest = false;
  
  // Package flags
  self.debugOnly = false;
  self.prodOnly = false;
  self.testOnly = false;
  self.devOnly = false;
  
  // Unibuilds - array of Unibuild objects
  self.unibuilds = [];
  
  // Plugins - map from plugin name to {arch -> JsImage}
  self.plugins = {};
  
  // Dependencies
  self.cordovaDependencies = {};
  self.isobuildFeatures = [];
  
  // Watch sets for rebuilding
  self.pluginWatchSet = new watch.WatchSet();
  
  // Loaded plugin state
  self.sourceProcessors = {
    compiler: null,
    linter: null,
    minifier: null
  };
};

Format Versions

Isopacks have evolved through multiple format versions:

Isopack.knownFormats = [
  "unipackage-pre2",  // Pre-0.9.0
  "isopack-1",         // Meteor 0.9.0+
  "isopack-2"          // Current format
];

Format Migration

The system can convert between formats:

// From isopack.js
Isopack.convertIsopackFormat = function (data, fromFormat, toFormat) {
  var fromPos = Isopack.knownFormats.indexOf(fromFormat);
  var toPos = Isopack.knownFormats.indexOf(toFormat);
  var step = fromPos < toPos ? 1 : -1;
  
  while (fromPos !== toPos) {
    if (step > 0) {
      data = Isopack.convertOneStepForward(data, fromFormat);
    } else {
      data = Isopack.convertOneStepBackward(data, fromFormat);
    }
    fromPos += step;
    fromFormat = Isopack.knownFormats[fromPos];
  }
  
  return data;
};

On-Disk Layout

When written to disk, an Isopack directory contains:

isopack.json

The main metadata file:

// From isopack.js - readMetadataFromDirectory
var isopackJsonPath = files.pathJoin(isopackDirectory, "isopack.json");

if (files.exists(isopackJsonPath)) {
  var isopackJson = JSON.parse(files.readFile(isopackJsonPath));
  
  if (isopackJson['isopack-2']) {
    metadata = isopackJson['isopack-2'];
  } else if (isopackJson['isopack-1']) {
    metadata = Isopack.convertIsopackFormat(
      isopackJson['isopack-1'], 'isopack-1', 'isopack-2');
  }
}

Unibuild Directories

Each unibuild has its own subdirectory with:

• Compiled JavaScript files
• CSS resources
• Source maps
• Assets
• Node modules

Unibuilds

From GLOSSARY.md:

Isopack is a package format containing source code for each architecture it can be ran on. Each separate part built for a separate architecture is called “Unibuild”.

There are multiple reasons why we can’t call it just “build” and historically the name “Unibuild” has been associated with parts of Isopacks.

Unibuild Properties

// Each Unibuild represents one architecture build
class Unibuild {
  constructor(isopack, options) {
    this.pkg = isopack;         // Parent isopack
    this.arch = options.arch;   // Target architecture
    this.uses = options.uses;   // Package dependencies
    this.implies = [];          // Implied packages
    this.watchSet = new watch.WatchSet();
    
    // Compiled resources
    this.resources = [];
    this.nodeModulesDirectories = {};
  }
}

Plugin Architecture

Isopacks can contain build plugins:

// Plugins are mapped by name and architecture
self.plugins = {
  'my-compiler': {
    'os': jsImageForOs,
    'os.linux.x86_64': jsImageForLinux
  }
};

Plugin Initialization

// From isopack.js
ensurePluginsInitialized: async function () {
  if (self._pluginsInitialized) {
    return;
  }
  
  // Initialize source processor sets
  self.sourceProcessors.compiler = 
    new buildPluginModule.SourceProcessorSet(
      self.displayName(), 
      { hardcodeJs: true, singlePackage: true }
    );
  self.sourceProcessors.linter = 
    new buildPluginModule.SourceProcessorSet(
      self.displayName(), 
      { singlePackage: true, allowConflicts: true }
    );
  self.sourceProcessors.minifier = 
    new buildPluginModule.SourceProcessorSet(
      self.displayName(), 
      { singlePackage: true }
    );
  
  // Load each plugin
  for (const [name, pluginsByArch] of Object.entries(self.plugins)) {
    var arch = archinfo.mostSpecificMatch(
      archinfo.host(), Object.keys(pluginsByArch));
    
    var plugin = pluginsByArch[arch];
    await plugin.load({
      Plugin,
      Profile,
      __meteor_bootstrap__
    });
  }
  
  self._pluginsInitialized = true;
}

Architecture Detection

The architectures() method returns all architectures in the package:

architectures: function () {
  var archSet = {};
  
  // Collect from unibuilds
  _.each(self.unibuilds, function (unibuild) {
    archSet[unibuild.arch] = true;
  });
  
  // Collect from plugins
  _.each(self.plugins, function (plugin, name) {
    _.each(plugin, function (plug, arch) {
      archSet[arch] = true;
    });
  });
  
  var arches = Object.keys(archSet).sort();
  
  // Remove 'os' if specific os.* arches exist
  if (_.any(arches, a => a.match(/^os\./))) {
    arches = _.without(arches, 'os');
  }
  
  return arches;
}

Tarball Naming

tarballName: function () {
  // Converts colons to safe characters
  return colonConverter.convert(self.name) + '-' + self.version;
}

Node Modules Integration

Isopacks track Node.js dependencies:

class NodeModulesDirectory {
  constructor({
    packageName,      // Package this belongs to
    sourceRoot,       // Root directory
    sourcePath,       // Absolute path to node_modules
    preferredBundlePath,  // Where to place in bundle
    local = false,    // Is it locally accessible?
    npmDiscards = null,
  }) {
    this.packageName = packageName;
    this.sourceRoot = sourceRoot;
    this.sourcePath = sourcePath;
    this.preferredBundlePath = preferredBundlePath;
    this.local = !! local;
    this.npmDiscards = npmDiscards;
  }
  
  isPortable() {
    return meteorNpm.dependenciesArePortable(this.sourcePath);
  }
}

Watch Set Integration

Isopacks maintain comprehensive watch sets for efficient rebuilding:

getSourceFilesUnderSourceRoot: function (sourceRoot) {
  var sourceFiles = {};
  
  // Collect from plugin watch set
  addSourceFilesFromWatchSet(self.pluginWatchSet);
  
  // Collect from each unibuild
  _.each(self.unibuilds, function (u) {
    addSourceFilesFromWatchSet(u.watchSet);
  });
  
  // Filter to files under sourceRoot
  _.each(watchSet.files, function (hash, filename) {
    if (!hash) return; // File doesn't exist
    
    var relativePath = files.pathRelative(sourceRoot, filename);
    if (relativePath.substr(0, 3) === '..' + files.pathSep) {
      return; // Not under sourceRoot
    }
    sourceFiles[relativePath] = true;
  });
  
  return Object.keys(sourceFiles);
}

Cordova Dependencies

Isopacks track Cordova plugin dependencies:

self.cordovaDependencies = {
  'cordova-plugin-camera': '2.4.1',
  'cordova-plugin-file': '6.0.1'
};

Package Metadata

self.metadata = {
  summary: "Package description",
  git: "https://github.com/user/repo",
  documentation: "README.md"
};

Storage Locations

• ~/.meteor/packages/ - Installed packages (release mode)
• APP/.meteor/local/isopacks/ - Local package builds
• REPO/.meteor/packages/ - Checkout mode packages

Related Topics

• Isobuild System - The build system
• Bundler - Creating bundles from isopacks
• Compiler Plugins - Build-time code transformation