Question

The endpoints of AB are A(0, 0) and B(6, 8)

Answer 1

We are given the endpoints. We are also given the endpoints after the transformation. For the first item, a simple distance formula verification would reveal that the distance between AB and A'B' is not equal. So, a dilation must have been done. Next, AB and A'B' are not parallel which means that a translation transformation must have been done.
After a dilation of 4/5 with A as the center, A' is still (0,0) and B' is (6 - 24/5, 8 - 32/5).