Help with manual webpack configuration for SFCs (vue-loader, Vue 2.7)

I am attempting to use Vue 2.7 composition API in my project. However, I am having problems with getting SFCs to compile with vue-loader.

As far as I can tell I have followed the information at these references:

Getting Started | Vue Loader (note that they gently contradict over whether vue-template-compiler is needed. It failed the same either way for me, though)

I have the following setup:

webpack.config.js

const { VueLoaderPlugin } = require("vue-loader");
const HtmlWebpackPlugin = require("html-webpack-plugin");

module.exports = {
  mode: "development",
  module: {
    rules: [
      {
        test: /\.vue$/,
        loader: "vue-loader",
      },
    ],
  },
  plugins: [new VueLoaderPlugin(), new HtmlWebpackPlugin()],
};

npm dependencies (note that I tried with explicitly including vue-demi as well since that is mentioned in the Vue 2.7 guide, but that did not help)

  "dependencies": {
    "vue-loader": "^15.10.0",
    "webpack": "^5.74.0",
    "webpack-cli": "^4.10.0"
    "html-webpack-plugin": "^5.5.0",
    "vue": "^2.7.8",
    "webpack-dev-server": "^4.10.0"
  }

main.js

import sfc from "./components/sfc.vue";

const target = document.createElement("div");
document.body.append(target);

new sfc({ el: target });

sfc.vue

<template>
  <div>
      I am a div!!
  </ div>
</ template>

The only thing that i can get working is explicitly using Vue.extend() (which the docs seems to recommend against)

<script>
import Vue from 'vue'
export default Vue.extend({})
</ script>

Any other attempt to define a component (script setup, export default {}, omitting script block) results in Uncaught TypeError: _components_sfc_vue__WEBPACK_IMPORTED_MODULE_0__.default is not a constructor at runtime.

For greater context, I am attempting to upgrade a codebase which currently makes use of TS and vue-property-decorator library to make use of some Composition API utilities. I am additionally having trouble with TS telling me that there are multiple default exports when I try to use script setup.