Can an offer show all products from a collection automatically? (Essential Upsell app)

No. Essential Upsell does not fill an offer's product list from a Shopify collection. An offer's products come from one of these sources:

• Products: you choose the products one by one, and you can choose a variant and set their order.
• Auto recommend: Shopify recommends products automatically. You choose the recommendation type, either Related products or Complementary products.

Collections are used in a different setting: where the offer appears. The trigger options are All products, Specific products, or Specific collections. Choosing a collection controls which products show the offer. It does not add that collection's products to the offer.