Preheat your oven to 375°F (190°C).
In a large pot, bring salted water to a boil. Add the diced potatoes and cook until tender, about 10-15 minutes. Drain and set aside.
In a large mixing bowl, combine the cooked chicken, broccoli, cheddar cheese, sour cream, milk, garlic powder, onion powder, salt, black pepper, and paprika. Mix well until all ingredients are evenly coated.
Gently fold in the cooked potatoes until well combined.
Transfer the mixture to a greased 9x13 inch baking dish. If using breadcrumbs, sprinkle them evenly over the top. Drizzle with olive oil if desired.
Bake in the preheated oven for 25-30 minutes, or until the casserole is heated through and the top is golden brown.
Remove from the oven and let it cool for a few minutes before serving.