Drupal 8 will standard give the term name as page title on /taxonomy/term/term-foo. I want to alter this so it says "Items tagged with Term foo". 

You should disable the title block and create a custom block in a module. Add a file in src/Plugin/Block named termpageTitle.php:

use Drupal\Core\Block\BlockBase;
use Drupal\Core\Cache\Cache;
/**
 * Provides a Block
 *
 * @Block(
 *   id = "termpagetitle",
 *   admin_label = @Translation("Term page title"),
 * )
 */
class termpageTitle extends BlockBase {
  /**
   * {@inheritdoc}
   */

  public function build() {
    return array(
      '#markup' => '',
    );
  }

  public function getCacheContexts() {
    return Cache::mergeContexts(parent::getCacheContexts(), ['url.path']);
  }

}

Then, in your mymodule.module file:

/**
 * Registers a template
 */
function mymodule_theme($existing, $type, $theme, $path) {
  return array(
    'block__termpagetitle' => array(
      'render element' => 'elements',
      'base hook' => 'block',
    )
  );
}

/**
 * Registers a variable to make it available in your twig file
 */
function mymodule_preprocess_block(&$variables) {
  if($variables['plugin_id'] == 'termpagetitle') {
    $request = \Drupal::request();
    if ($route = $request->attributes->get(\Symfony\Cmf\Component\Routing\RouteObjectInterface::ROUTE_OBJECT)) {
      $title = \Drupal::service('title_resolver')->getTitle($request, $route);
    }
    $variables['title'] = $title['#markup'];

    return $variables;
  }
}

Now, create a file in templates folder of your module named block--termpagetitle.html.twig:

<h1 class="title">
  Items tagged with <i>{{ title }}</i>
</h1>

Add the block to the proper region and urls and enjoy!